Jack used the compound as a base for his successful 1960 U.S. Presidential campaign and later as a summer White House and presidential retreat until his assassination in 1963. Ted died at the compound in 2009 and in 2012 the main house was donated to the Edward M. Kennedy Institute for the United States Senate, which said the house would eventually be opened to the public.

In 1926 Joseph P. Kennedy rented a summer cottage in Hyannisport. The house has changed little, either structurally or in furnishings, since John's association with it. Two years later, he purchased the structure, which had been erected in 1904, and enlarged and remodeled it to suit his family's needs. In 1956, Jack bought a smaller home nearby. Ted later also bought a house in the area. All three buildings are white-frame clapboard structures typical of vacation residences on Cape Cod. Except for specific occasions at the Main House, the buildings are not available for public visitation.

Joe's home, the Main House and the largest of the three, is surrounded by well-tended lawns and gardens and it commands sweeping views of the ocean from its long porches. On the main floor are a living room, dining room, sun room, television room, the bedroom that John used before he purchased his own house in the compound, the kitchen, and various pantries and utility rooms. On the second floor are six bedrooms, a sewing room, packing room, and four servants' bedrooms. The house has a full attic.

The basement contains a motion-picture theater and a hall covered with dolls from all around the world. A wine cellar designed after a ship's hull and a sipping room – one of the Kennedy family's favorite hideouts. It is considered the place that Ted coined the well-known toast "There are good ships, and there are wood ships, the ships that sail the sea. But the best ships are friendships, and may they always be."