Caldwell was born on Caldwells Run in Ohio County near Wheeling to Mary Yarnell Caldwell (1781-1819) and her husband Joseph Caldwell (1777 - 1864). His grandfather, James Caldwell (1724-1804), emigrated from northern Ireland with his family and in 1770 settled in the near-wilderness that became Ohio County. With Ebenezer Zane, he helped found the town of Wheeling on the Ohio River, becoming a justice of the peace in 1777 and militia commander during the American Revolutionary War, as well as raising a large family. His sons (Bolton's uncles) James Caldwell (Ohio politician) and Alexander Caldwell (Virginia) helped found the state of Ohio (and became the first U.S. Congressman for the district across the river from Wheeling) and U.S. District Judge for the Western District of Virginia, respectively.
His father Joseph, the youngest son of James Caldwell Sr., after military service during the War of 1812 became the first president of Wheeling's Merchants and Merchants Bank. He remarried after Mary died when young Bolton was five. Thus, Bolton had an older sister Anne McGill Caldwell Moyston (1805 - 1865) and a younger full brother, James Yarnall Caldwell (1815 - 1853), as well as younger half brothers Henry Clay Caldwell (1832 - 1917) and Alexander J. Caldwell (1835 - 1916).
After private education locally, Bolton attended Jefferson College (now Washington & Jefferson College) and received a degree in 1833, shortly before his cousin Alfred Caldwell (politician) (son of James Caldwell Jr.). Bolton Caldwell then went to Harvard University and studied law under Joseph Story and Greenleaf, as his cousin Alfred would soon also emulate.
On May 14, 1846, A. Bolton Caldwell married Mathilda Anne Newman (1824-1906), with whom he would have Therese Caldwell Moise (1847-1882) and George E. Caldwell (1863-1866).
After graduation, Bolton Caldwell sought his fortune by practicing law in Mississippi beginning in 1840. He moved to Louisiana in 1844, won elections in 1845, 1847 and 1852 and became the parish attorney (prosecutor) for East Carroll Parish, Louisiana. Caldwell then returned home and joined the Virginia bar. In 1858 he returned to Wheeling (where his cousin Alfred had begun his own legal practice and already twice been elected mayor and once to the Virginia Senate), and began his legal practice, which included stints as a federal prosecutor and as the Ohio County prosecutor as discussed below.
A. Bolton Caldwell became the United States Attorney for the Western District of Virginia after nomination by President Abraham Lincoln and the concurrence of the U.S. Senate, serving from 1861 until 1863. His cousin Alfred Caldwell, meanwhile, as a reward for his active support of President Lincoln and the Republican Party, was appointed consul to the Kingdom of Hawaii in 1861 (and with the Senate's approval served until 1867, when he was removed for malfeasance by President Johnson).
Meanwhile, in 1863 West Virginians elected Bolton Caldwell as their first Attorney General. He served for 18 months, completing his term. During President Lincoln's first term in office, Caldwell was offered an appointment as Judge to the United States District Court, but for personal reasons, he turned down the appointment, John Curtiss Underwood being selected instead.
On July 1, 1869 Caldwell returned to the Office of the West Virginia Attorney General when he was appointed by the Governor after Attorney General Thayer Melvin resigned. After his term as Attorney General Bolton Caldwell won election as Ohio County Prosecutor and later Circuit Judge.
Caldwell died in 1893 at the home of his son-in-law in Wheeling, and was buried in Greenwood Cemetery. His cousin Alfred Caldwell Jr. (his cousin and ally Alfred Caldwell's son) would become West Virginia's 10th attorney general, first winning election as a Democrat in 1885.