Madison was born into slavery at Monticello, where his mother Sally Hemings was a mixed-race slave inherited by Martha Wayles Skelton, the wife of Thomas Jefferson. (Sally and Martha were reported half sisters, both fathered by the planter John Wayles. Wayles was said to have a "shadow family": six children with his slave, Betty Hemings, whom he took as a sex slave after his third wife died.) As the historians Philip D. Morgan and Joshua D. Rothman have written, there were numerous interracial relationships in the Wayles-Hemings-Jefferson families, Albemarle County and Virginia, often with multiple generations repeating the pattern. According to his memoir, Sally Hemings told Madison that his father was Thomas Jefferson, and that their relationship had started in Paris in the late 1780s, where he was serving as a diplomat. Pregnant, she agreed to return with Jefferson to the United States based on his promise to free her children when they came of age.
Madison grew up at Monticello. His surviving mixed-race siblings were an older brother Beverly and sister Harriet, and a younger brother Eston. According to his 1873 memoir, Madison was named for Jefferson's close friend and future president James Madison at the request of Madison's wife Dolley. Madison lived as a child with his siblings and mother, who were all spared from hard labor. He described Jefferson as kind but showing little or no paternal interest in the Hemings' children.
Like his older brother Beverley, at 14 years of age, Madison was apprenticed to his uncle, Sally's brother John Hemings, the most skilled artisan at Monticello, to learn carpentry and fine woodworking; his younger brother Eston joined him two years later. This gave each of them a valuable trade. All three of the Hemings brothers also studied and learned to play the violin, the instrument associated with Jefferson. Beverley, the oldest, was good enough to be invited to play at dances held by the Jeffersons at Monticello. As an adult, Eston Hemings made a living as a musician and entertainer in Ohio.
In his will, Jefferson gave immediate freedom to three slaves: John Hemings, a brother of Sally, to whom he also bequeathed "the service of his two apprentices Madison and Eston Hemings", with instruction that the brothers each be freed at his respective 21st birthday. Jefferson freed two of Sally's nephews: Joseph Fossett and Burwell Colbert. (John Hemings was a widower and evidently childless by 1826, but Fossett and Colbert were married and the fathers of large families. As Jefferson did not free their wives and children, all were sold along with Monticello's nearly 130 other slaves at auctions in 1827 to settle the heavy debts against his estate. The men and their friends worked to buy the freedom of their families.) Although the three older men had served Jefferson for decades, Madison and Eston were distinguished by being freed as they "came of age" at 21. Madison was nearly 21 at the time of Jefferson's death; Eston was "given his time" and freed before age 21.
Knowing that his estate was in debt and that freed slaves could not legally remain in Virginia for more than one year, Jefferson by his will requested the legislature of Virginia to guarantee the manumission of the five slaves, and to grant the men special "permission to remain in this State, where their families and connections are." Both requests were evidently granted.
Twenty-one-year-old Madison Hemings was emancipated almost immediately after Jefferson died; Eston soon after. The brothers rented a house in nearby Charlottesville, where their mother Sally joined them for the rest of her life. (She was not formally freed but was "given her time" by Jefferson's surviving daughter Martha Randolph, who was also Hemings' niece). In the 1830 Albemarle County census, Madison, Eston and Sally Hemings were all classified as free whites.
According to Madison's 1873 memoir, his older brother Beverley and his older sister Harriet moved to Washington D.C. in 1822 when they "ran away" from Monticello. Jefferson ensured that Harriet was given money for her journey. Because of their light skin and appearance (they were 7/8 European or octoroon), both identified with the white community after their moves and probably changed their names. Hemings said they had married white spouses of good circumstances, and moved into white society. They apparently kept their paternity a secret, as it would have revealed their origins as slaves, and disappeared into history.
In September 1831, in his mid-twenties, Madison Hemings was described in a special census of the State of Virginia as being: 5"7 3/8 Inches high light complexion no scars or marks perceivable". Forty-two years later at the time of his interview, a journalist described him as "five feet ten inches in height, sparely made, with sandy complexion and a mild gray eye."
In 1834 Madison wed Mary Hughes McCoy, a free woman of mixed-race ancestry (her grandfather Samuel Hughes, a white planter, freed her grandmother Chana from slavery and had children with her.) They had two children born in Virginia.
In 1836 Madison, Mary and their infant daughter Sarah left Charlottesville for Pike County, Ohio, probably to join his brother Eston, who had already moved there with his own family. They lived in Chillicothe, which had a thriving free black community, abolitionists among both races, and a station of the Underground Railroad. Surviving records in Pike County state that Hemings purchased 25 acres (100,000 m2) for $150 on July 22, 1856, sold the same area for $250 on December 30, 1859, and purchased 66 acres (270,000 m2) for $10 per acre on September 25, 1865. The Hemings had more children born in Ohio.
In 1852, Madison's brother, Eston, moved with his family away from Ohio (and his brother) to Madison, Wisconsin, to get further from possible danger due to passage of the Fugitive Slave Act of 1850. Slave catchers were known to kidnap free blacks and sell them into slavery, as demand and prices were high in the Deep South. In Wisconsin, the family all took the surname Jefferson and entered the white community. They lived according to their appearance and mostly white ancestry. Their oldest son John Wayles Jefferson served as a Union officer in the American Civil War, and was promoted to colonel. Their son Beverly also served in the Union Army and married a white woman. Their daughter Anna married a white man. All of Eston's descendants identified as white.
In 1873, Madison used an Ohio newspaper interview, titled, "Life Among the Lowly," to address the Jefferson/Hemings controversy, where he stated that Jefferson was his father. In this interview, Madison also states, "I was named Madison by the wife of James Madison, who was afterwards President of the United States. Mrs. Madison happened to be at Monticello at the time of my birth, and begged privilege of naming me, promising my mother a ﬁne present for the honor. She consented, and Mrs. Madison dubbed me by the name I now acknowledge, but like many promises of white folks to the slaves she never gave my mother anything."
Madison and Mary Hemings were the parents of 10 surviving children. According to his memoir, their daughter Sarah (named for his mother) and an unnamed son who died in infancy were born in Virginia; nine more children were born in Ohio. He had a quiet life as a modestly successful free black farmer and carpenter.
The Jefferson–Hemings controversy concerns the question of whether, after Jefferson became a widower, he had an intimate relationship with his mixed-race slave, Sally Hemings and fathered her children. The controversy dates from the 1790s, and newspaper articles appeared during Jefferson's lifetime accusing him of fathering children with a slave named Sally. In the late twentieth century historians began reanalyzing the body of evidence. In 1997, Annette Gordon-Reed published a book that analyzed the historiography of the controversy, demonstrating how historians since the nineteenth century had accepted early assumptions and failed to note all the facts. In this book, Annette Gordon-Reed discusses many important facts that other writers of the Jefferson–Hemings controversy were too biased to support. The most convincing evidence is that Sally Hemings' children either ran away without being pursued or were granted their freedom by Jefferson. In his memoirs, Madison Hemings wrote that Jefferson did not display fatherly warmth to the Hemings children, but he did give them lighter work responsibilities compared to other slaves.
Sally Hemings had at least five children whose births were recorded. Some sources, including Madison Heming's memoir, also state that Sally Hemings conceived her first child while in Paris with Jefferson, but the baby died shortly after birth. Another daughter named Harriet, whose birth was recorded at the time, also died shortly after birth, but four others lived to adulthood, three boys and one girl: Beverly, Harriet (the second daughter given this name), Madison, and Eston. Beverly and Harriet left Monticello when they were both around twenty-one years of age, but Madison and Eston were freed by Jefferson's will after he died. Although Jefferson did not legally manumit Beverly and Harriet, he secretly arranged and paid for Harriet's transportation to Philadelphia, using his overseer Edmund Bacon as an intermediary. Although he marked in his Farm Book that both had "run away," Jefferson never made any attempt to re-enslave them. No one had ever heard of Jefferson letting slaves be free in this way, so why these particular ones? A consensus began to emerge after the results of a DNA analysis in 1998, which showed no match between the Carr male line, proposed for more than 150 years as the father(s), and the one Hemings descendant tested. It did show a match between the rare Y-DNA haplotype of the Jefferson male line and the Hemings descendant.
Since 1998 and the DNA study, which affirmed historical evidence, many historians have accepted that the widower Jefferson had a long, intimate relationship with Hemings, and fathered six children with her, four of whom survived to adulthood. The Thomas Jefferson Foundation (TJF), which runs Monticello, conducted an independent historic review in 2000, as did the National Genealogical Society in 2001; the scholars of both reviews concluded Jefferson was probably the father of all Hemings's children.
Critics, such as the Thomas Jefferson Heritage Society (TJHS) Scholars Commission (2001), have argued against these conclusions. They have concluded that there is insufficient evidence to determine that Jefferson was the father of Hemings's children. The TJHS report suggested that Jefferson's younger brother Randolph Jefferson could have been the father, and that Hemings may have had multiple partners.
There are no living male-line descendants of Madison Hemings, and Beverley Hemings' descendants have been lost to history. Descendants of Madison Hemings declined to have the remains of his son William Hemings disturbed to extract DNA for testing (he was buried in a VA cemetery), just as Wayles-Jefferson descendants declined to have Thomas Jefferson's remains disturbed.
In 2012, the Smithsonian Institution and the Thomas Jefferson Foundation held a major exhibit at the National Museum of American History: Slavery at Jefferson's Monticello: The Paradox of Liberty; it says that "evidence strongly support[s] the conclusion that Jefferson was the father of Sally Hemings' children." The exhibit explores the lives of six major slave families, including the Hemings, starting with the matriarch Elizabeth Hemings, who had 75 descendants at Monticello.
The Hemingses' youngest daughter Ellen Wayles Hemings married Andrew Jackson Roberts, a graduate of Oberlin College. They moved from Ohio to Los Angeles, California in 1885 with their first son Frederick, age six. The senior Roberts founded the first black-owned mortuary there and became a civic leader in the developing community. Their son, Frederick Madison Roberts, named for his maternal grandfather, was college-educated and became a businessman in partnership with his father. He also became a community leader.
In 1918 he was first elected to the California legislature. He was re-elected numerous times, serving for a total of 16 years, and becoming known as "dean of the assembly". He is believed to have been the first person of African-American ancestry elected to political office west of the Mississippi River. Both he and his brother William Giles Roberts graduated from college. The Roberts descendants for generations have had a strong tradition of college education and public service.
"The experiences of descendants of both Madison and Eston Hemings illustrate the benefits and costs of passing for white. None of Madison Hemings's sons married. William Beverly Hemings served in a white regiment--the 73rd Ohio--in the Civil War and died alone in a Kansas veterans hospital in 1910. His brother James Madison Hemings seems to have slipped back and forth across the color line, and may be the source of stories among his sisters' descendants of a mysterious and silent visitor who looked like a white man, with white beard and blue eyes. Several of Madison Hemings's grandsons also passed for white, divorcing themselves from their sisters who stayed on the other side of the line.
Passing was not always permanent. Intermittent passing became a strategy for securing anything from a job to a haircut. Their racial identities calibrated by the day or hour, light-skinned members of the Hemings family were white in the workplace and black at home, or they borrowed a white surname to make a hairdressing appointment in a neighboring town."
Many of the Hemings' descendants who remained in Ohio were interviewed in the late twentieth century by two Monticello researchers as part of the Thomas Jefferson Foundation's "Getting Word" project. They were collecting oral histories from the descendants of slave families at Monticello; material has been added to the Monticello website and was included in the national Slavery at Jefferson's Monticello 2012 exhibit. The researchers found that Hemings' descendants had married within the mixed-race community for generations, choosing light-skinned spouses of an educated class and identifying as people of color within the black community.
In 2010 Shay Banks-Young and Julie Jefferson Westerinen, descendants of Sally Hemings who identify as black and white, respectively, were honored together with David Works, a descendant of Martha Wayles Skelton Jefferson, with the Search for Common Ground award for "their work to bridge the divide within their family and heal the legacy of slavery." They have spoken about race and their historically divided and united family, and have been featured on NPR and in other interviews across the country.Delilah L. Beasley, Negro Trail Blazers of California, Los Angeles: 1919, pp. 137, 215-16. (An early picture of Roberts appears on p. 40.)Fawn M. Brodie, Thomas Jefferson: An Intimate History, New York: W.W. Norton & Co., 1974Annette Gordon-Reed, The Hemingses of Monticello: An American Family, New York: W.W. Norton & Co., 2008Shannon Lanier and Jane Feldman, Jefferson's Children: The Story of One American Family New York: Random House Books for Young Readers, 2000 (with photos of Jefferson descendants on both sides)Stanton, Lucia. Free Some Day: The African-American Families of Monticello, Charlottesville: Thomas Jefferson Foundation, 2000.