The house is believed to have been built in the early 1840s by a family named Bowen. Alphonso bought the house , with its accompanying 1.82 acres, for $10,000 on June 13, 1851. The residence, a Greek Revival domicile, was relatively modest compared to other nearby residences.
By 1961, the house was in poor condition and needed restoration, to the tune of $92,500. It was declared a National Historic Landmark in 1964. A ceremony on September 15, 1964 (William's 107th birthday), celebrated the home becoming a National Historic Landmark. The association gained full title to the house in 1968 and in 1969 transferred it to the National Park Service, which currently operates the site as a historic house museum, so that its future upkeep is ensured. The US government took the property title on November 1, 1970.