The Carter Library and Museum includes some parts that are owned and administered by the federal government, and some that are privately owned and operated. The library and museum are run by the National Archives and Records Administration and are part of the Presidential Library system of the federal government. Privately owned areas house Carter's offices and the offices of the Carter Center, a non-profit human rights agency.

The building housing the library and museum makes up 69,750 square feet, with 15,269 square feet of space for exhibits and 19,818 square feet of archive and storage space. The library stacks house 27 million pages of documents; 500,000 photos, and 40,000 objects, along with films, videos, and audiotapes. These collections cover all areas of the Carter administration, from foreign and domestic policy to the personal lives of President and Mrs. Carter.

The library was built in the Poncey-Highland neighborhood of Atlanta, on land that had been acquired by the state of Georgia DOT, for an interchange between two redundant highways that were cancelled by Carter when he was governor of Georgia, in response to the Atlanta freeway revolts. Construction started on October 2, 1984, and the library was opened to the public on Carter's 62nd birthday, October 1, 1986. A $10 million renovation of the museum began in April 2009 with completion on President Carter's 85th birthday in October 2009.