Graves was born in Hope Hull, Alabama, son of David and Mattie Bibb Graves and a descendant of Alabama's first governor William Wyatt Bibb. Graves' father died when he was one year old, and he was reared first by his paternal grandfather on an Alabama farm and then by an uncle in Texas. Graves attended the University of Alabama, where he was a member of the school's inaugural football team. After graduating with a degree in civil engineering (1893), Graves earned a degree from Yale Law School (1896). Graves was then elected to the Alabama legislature and later served as the city attorney in Montgomery.
As adjutant general of the Alabama National Guard, he helped organize the 1st Alabama Cavalry and served on the Mexican border in 1916. In World War I, Graves commanded as a colonel the 117th U.S. Field Artillery in France, and upon his return to Alabama, he helped organize the state section of the American Legion.
Graves lost his first campaign for governor in 1922, but four years later, with the secret endorsement of the Ku Klux Klan, he was elected to his first term as governor. Almost certainly Graves was the Exalted Cyclops (chapter president) of the Montgomery chapter of the Klan, but both Graves and U.S. Supreme Court Justice Hugo Black, another Alabama Klan member, were more opportunists than ideologues, politicians who used the temporary strength of the Klan to further their careers. After receiving solid gold "passports" from the Klan, Graves and Black were collectively known in some Alabama circles as "The Gold Dust Twins."
As governor, Graves earned a reputation as a reformer, abolishing the convict leasing system and raising taxes on public utilities, railroads, and coal and iron companies. The new revenue was used to expand educational and public health facilities, increase teachers' salaries and veterans' pensions, fund an ambitious road-building program, and improve port facilities in Mobile. "To maintain his popularity among the farmers in northern Alabama and the working classes, Graves made good on his commitment to New Deal legislation, winning a reputation as one of the most progressive governors in the South." During his second gubernatorial administration he supported Franklin D. Roosevelt's “court packing” plan and Hugo Black's nomination to the U.S. Supreme Court. In 1937, when Black's ties to the Klan were debated in Congress, Graves noted his own previous membership as well, a membership that had been publicly revealed when he resigned from the organization in 1928.
Graves appointed his own wife, Dixie Bibb Graves, to serve the remainder of Black's term. She thus became Alabama's first woman U.S. senator.
Graves made many successful trips to Washington to secure funds for Alabama, which he called "plum-tree-shaking expeditions," and President Roosevelt appointed him to a national advisory committee on agriculture and to an interregional highway committee. Graves was a strong opponent of eugenic sterilization, and by 1938, he was on hand to greet the 1,200 delegates to the founding session of the Southern Conference on Human Welfare—a fifth of whom were black.
Graves was a member of Phi Beta Kappa and an elder of the Christian Church. He was a founding member of the board of trustees of Bob Jones College and a personal friend of the founder, evangelist Bob Jones, Sr. Bibb Graves died in Sarasota, Florida while preparing for another gubernatorial campaign.
The University of Montevallo has a Bibb Graves Hall, the University of West Alabama has a Bibb Graves Hall and a Bibb Graves Auditorium, the University of North Alabama has a Bibb Graves Hall, Auburn University has both a Bibb Graves Amphitheatre and a Bibb Graves Drive, the School of Education at the University of Alabama is named for him, and the oldest classroom building at Troy University that houses the Sorrell College of Business is named Bibb Graves Hall. The historically black Alabama A&M University has a Bibb Graves Hall, and the historically black Alabama State University has a women's dormitory named Bibb Graves Hall. Jacksonville State University also has an administrative building named Bibb Graves Hall. Bob Jones University had a residence hall named for Graves until 2011, when it was renamed for H. A. Ironside. Bibb Graves High School in Millerville, Clay County, Alabama, closed in 2003. A bridge in Wetumpka, Elmore County, Alabama, built in 1937, is named for Graves.