James T. Rapier was born free in 1837 in Florence, Alabama to John H. Rapier, a prosperous local barber, and his wife, who were established free people of color. As a youth, he went to Nashville, Tennessee to attend an academy for high school, and lived with his grandmother, an enslaved clothes cleaner.
In 1856 his father sent James to Canada for further education. He first settled in North Buxton, Ontario, an all-black community developed with the aid of Rev. William King, a Scots-American Presbyterian missionary. King had bought land (with government approval) for resettlement of black American refugees who had escaped to Canada during the slavery years via the Underground Railroad. The African Americans were building a thriving community, and Rapier's uncle had property there. Rapier attended the Buxton Mission School, which was highly respected. He went on to Montreal College, where he studied law. He also went to Scotland, where he studied at the University of Glasgow. After returning to the United States, Rapier attended Franklin College, a historically black college in Nashville, Tennessee and obtained a teaching certificate in 1863.
Rapier started working for black suffrage in Tennessee in 1865 and attended the state's suffrage convention that year. He bought real estate and became a cotton planter and an advocate for black voting rights. Disappointed at the lack of progress in Tennessee, he returned to his home in Alabama in 1866. He cultivated cotton and became active in the Republican Party. He served as a delegate to the 1867 state constitutional convention.
In 1870 Rapier ran for Alabama Secretary of State and lost. In 1872 he was elected to the Forty-third United States Congress from Alabama's 2nd congressional district, one of three African-American congressmen elected from the state during Reconstruction. While in Congress, he had national scope and proposed authorizing a land bureau to allocate Western lands to freedmen. He also proposed that Congress appropriate $5 million to devote to public education in Southern schools.
He was one of seven black Congressmen at the time; in 1874 they each testified for the Civil Rights Act, which was signed in 1875. Rapier recalled being denied service at every inn at stopping points between Montgomery and Washington, DC. He noted how the race issue in the United States society related to class and religious inequalities in other lands, and said that he was "half slave and half free", having political rights but no civil rights. He said that in Europe, "they have princes, dukes, and lords; in India, "brahmans or priests, who rank above the sudras or laborers;" in America, "our distinction is color."
After losing his re-election campaign in 1874, Rapier was appointed by the Republican presidential administration as a collector for the Internal Revenue Service, serving in this role until his death. He campaigned against the conservative Democratic Party's Redeemer government in Alabama, but Democrats regained control of the state legislature in 1874. After passing other restrictive laws, in 1901 they passed a new constitution that required poll taxes and literacy tests for persons trying to register to vote. As they were administered, they disenfranchised most blacks, excluding them from the political system.
He died in Montgomery, Alabama on May 31, 1883 of pulmonary tuberculosis. He was buried in Calvary Cemetery in St. Louis, Missouri.
In 1979 historian John Hope Franklin gave a presidential address  to the American Historical Association. He noted how Walter L. Fleming of Vanderbilt University, one of the most prominent of the influential 20th-century Dunning School historians, wrote about Rapier. Franklin observed that Fleming's viewpoint, which had been hostile to civil and voting rights for African Americans, may have led him to make errors.
Writing in 1905 Walter L. Fleming referred to James T. Rapier, a Negro member of the Alabama constitutional convention of 1867, as "Rapier of Canada." He then quoted Rapier as saying that the manner in which "colored gentlemen and ladies were treated in America was beyond his comprehension."[Fleming, Civil War and Reconstruction in Alabama
] In a footnote to his address, Franklin added: "Fleming knew better, for in another place—deep in a footnote (p. 519)—he asserted that Rapier was from Lauderdale, "educated in Canada"."
Born in Alabama in 1837, Rapier, like many of his white contemporaries, went North for an education. The difference was that instead of stopping in the northern part of the United States, as, for example, (the pro-slavery advocate) William L. Yancey did, Rapier went on to Canada. Rapier's contemporaries did not regard him as a Canadian; and, if some were not precisely clear about where he was born (as was the Alabama State Journal
, which referred to his birthplace as Montgomery rather than Florence), they did not misplace him altogether. [Loren Schweninger, James T. Rapier and Reconstruction
(Chicago, 1978), xvii, 15.]
Franklin said: "In 1905 Fleming made Rapier a Canadian because it suited his purposes to have a bold, aggressive, 'impertinent' Negro in Alabama Reconstruction come from some non-Southern, contaminating environment like Canada. But it did not suit his purposes to call Yancey, who was a graduate of Williams College, a 'Massachusetts Man.' Fleming described Yancey (a white Confederate) as, simply, the 'leader of the States Rights men.'" [Fleming, Civil War and Reconstruction in Alabama, p. 12.] For a detailed account and comparison of Yancey and other white Southerners who went North to secure an education, see Franklin's book, A Southern Odyssey: Travelers in the Antebellum North (Baton Rouge, Louisiana State University Press, 1976), pp. 45–80.
Franklin is critical of Fleming for falsely stating that Rapier, and others, were "carpetbaggers." Franklin said, "...some of the people that Fleming called carpetbaggers had lived in Alabama for years and were, therefore, entitled to at least as much presumption of assimilation in moving from some other state to Alabama decades before the war as the Irish were in moving from their native land to some community in the United States. ...Whether they had lived in Alabama for decades before the Civil War or had settled there after the war, these "carpetbaggers" were apparently not to be regarded as models for Northern investors or settlers in the early years of the twentieth century. Twentieth-century investors from the North were welcome provided they accepted the established arrangements in race relations and the like. Fleming served his Alabama friends well by ridiculing carpetbaggers, even if in the process he had to distort and misrepresent."