His father was Judge Peter Van Ness (1734-1804), who owned most of the land which Martin Van Buren later purchased for construction of the Lindenwald estate.
He was rumored to have lost $5000, and with it, his father's home, Lindenwald as well as a mistress, the very popular Elena America Vespucci, descendent of Amerigo Vespucci, to George Parish of Ogdensburg, New York in a card game at the LeRay Hotel in Evans Mills, New York.
Van Buren named the estate Lindenwald, which is German for "linden wood", after the American Linden (American Basswood or Tilia americana) trees lining the Albany-to-New York Post Road, which is still located in front of the home.