During a weekend visit to Boston on October 19, 1963, President Kennedy, along with John Carl Warnecke — the architect who would design the President’s tomb in Arlington — viewed several locations offered by Harvard as a site for the library and museum. Kennedy chose a plot of land next to the Harvard Graduate School of Business Administration.

Since Kennedy encouraged his administration to save effects of both personal and official nature, the complex would not just be a collection of the President's papers, but "a complete record of a Presidential era." And so, the building would have the word museum appended to its name: John F. Kennedy Presidential Library and Museum.

On December 13, 1964, the Kennedy family announced that I.M. Pei was unanimously chosen by a subcommittee as the architect of the library. The library's first floor features a museum containing video monitors, family photographs, political memorabilia. Visitors to the museum begin their visit by watching a film narrated by President Kennedy in one of two cinemas that show an orientation film, and a third shows a documentary on the Cuban missile crisis.

The are seven permanent exhibits: Campaign Trail, The Briefing Room, The Space Race, Attorney General's Office, The Oval Office, First Lady Jacqueline Bouvier Kennedy, and The Kennedy Family.

Among the Library's art collection is a 1962 portrait of Robert F. Kennedy by Lajos Markos, a watercolor sketch of John F. Kennedy by Jamie Wyeth, a watercolor painting of the White House painted by Jacqueline Kennedy and given as a gift to her husband, who had it hung in the Oval Office, a fingerpainting by Caroline Kennedy as a child, and a bust of John F. Kennedy sculpted by Felix de Weldon.