According to family history, Smith was born into slavery, the son of Susan, a slave, and Elijah Smith, a slave owner. Upon the outbreak of the Civil War, Elijah Smith joined the Confederate military, with the intention of taking 19-year-old Andrew along with him. When Andrew Smith learned of this, he and another slave ran away, walking 25 miles (40 km) through the rain before presenting themselves to a Union Army regiment, the 41st Illinois Volunteer Infantry Regiment, in Smithland, Kentucky.
Smith was taken in by the 41st Illinois and became a servant to Major John Warner at the regiment's post in nearby Paducah, Kentucky. Among Smith's duties were, in the event of Warner's death, to return his belongings to his home in Clinton, Illinois. On March 10, 1862, the regiment moved out to Pittsburg Landing, Tennessee, where it took part in the Battle of Shiloh a month later. During the fighting, Smith supplied Major Warner with fresh horses after the officer had two mounts shot out from under him. Smith was then struck by a "spent minie ball that entered his left temple, rolled just under the skin, and stopped in the middle of his forehead." The bullet was removed by the regimental surgeon, leaving Smith with only a scar.
By November 30, 1864, Smith was serving as a corporal in the 55th Massachusetts Volunteer Infantry Regiment. On that day, both the 55th and its sister regiment, the 54th Massachusetts Volunteer Infantry Regiment, participated in the Battle of Honey Hill in South Carolina. The two units came under heavy fire while crossing a swamp in front of an elevated Confederate position. When the 55th's color bearer was killed, Smith took up the battle flags and carried them through the remainder of the fight. It was for this action that Smith was later awarded the Medal of Honor.
Smith was promoted to color sergeant before leaving the Army. After the war, he lived in Kentucky, where he bought and sold land. He died at age 88 and was buried in Mount Pleasant Cemetery, Grand Rivers, Kentucky.
Smith was nominated for the Medal of Honor in 1916, but the Army denied the nomination, citing a lack of official records documenting his case. Smith's commander at Honey Hill had not included an account of Smith's actions in the official battle report. It was not until January 16, 2001, 137 years after the Battle of Honey Hill, that Smith was recognized; President Bill Clinton presented the Medal of Honor to several of Smith's descendants during a ceremony at the White House on that day. Former President Theodore Roosevelt was also posthumously awarded the medal at the same ceremony, for his actions during the Spanish–American War.
Smith's official Medal of Honor citation reads:
Corporal Andrew Jackson Smith, of Clinton, Illinois, a member of the 55th Massachusetts Voluntary Infantry, distinguished himself on 30 November 1864 by saving his regimental colors, after the color bearer was killed during a bloody charge called the Battle of Honey Hill, South Carolina. In the late afternoon, as the 55th Regiment pursued enemy skirmishers and conducted a running fight, they ran into a swampy area backed by a rise where the Confederate Army awaited. The surrounding woods and thick underbrush impeded infantry movement and artillery support. The 55th and 54th regiments formed columns to advance on the enemy position in a flanking movement. As the Confederates repelled other units, the 55th and 54th regiments continued to move into flanking positions. Forced into a narrow gorge crossing a swamp in the face of the enemy position, the 55th's Color-Sergeant was killed by an exploding shell, and Corporal Smith took the Regimental Colors from his hand and carried them through heavy grape and canister fire. Although half of the officers and a third of the enlisted men engaged in the fight were killed or wounded, Corporal Smith continued to expose himself to enemy fire by carrying the colors throughout the battle. Through his actions, the Regimental Colors of the 55th Infantry Regiment were not lost to the enemy. Corporal Andrew Jackson Smith's extraordinary valor in the face of deadly enemy fire is in keeping with the highest traditions of military service and reflect great credit upon him, the 55th Regiment, and the United States Army.