Johnson was born to William D. Johnson and the former Sarah J. Slaughter in Dadeville in Tallapoosa County in east central Alabama. His father fought for the Confederacy in the American Civil War. Before the outbreak of hostilities in 1861, Johnson had already moved his family to a plantation in Scott County near Forest in central Mississippi. There, Andrew was educated in public schools and at Harperville Academy in the community of Harperville (not to be confused with Harpersville, Alabama). When not in school, he assisted his father on the farm. Young Andrew often walked five miles to school.
While in his early twenties, Johnson moved to Columbia County near Magnolia in southern Arkansas. He was a schoolteacher in Arkansas, clerked in a store, and served as the mayor of an unnamed town. He was also employed as depot agent by the Cotton Belt Railroad and edited a newspaper. He held some of these positions simultaneously. He also went into business for himself as a merchant. On April 18, 1883, he married the former Julia C. Pittman. The couple had eleven children, eight of whom were still living in 1925.
In the middle 1890s, Johnson was employed in Wisconsin and Oklahoma for three years by the U.S. government as special agent of the United States Land Office. He left government service and in 1897 moved with his family to Homer, the seat of Claiborne Parish in north Louisiana. A year later, he moved to northern Natchitoches Parish, where he sold lots in the new village of Ashland, formed in 1901. Johnson named Ashland for his former residence in Ashland, Wisconsin, where he had worked for the Land Office.
A resident of the Ashland area for eight years from 1898 to 1906, he was engaged in the timber and real estate businesses. He returned to Homer in 1906, where he became president of the Homer State Bank, served on the Claiborne Parish School Board, and was twice elected mayor, probably in 1910 and 1914. During his tenure as mayor, Homer acquired electric lights and water works. At the time Homer already had a municipal home-rule charter.
While Johnson was in Ashland, the since disbanded Louisiana and Arkansas Railroad, which linked Hope, Arkansas, with Alexandria, Louisiana, came through the area. Johnson donated the land on which the since defunct Ashland High School was constructed in 1907. The structure burned in 1918 and was rebuilt by 1919.
In 1913, Johnson was a Claiborne Parish delegate to the Louisiana Constitutional Convention. (There were subsequent conventions in 1921 and 1973.) He was elected without opposition for both Senate terms and served on the Senate Finance Committee. As his second term wound down, Johnson considered entering the Democratic primary election for governor in 1924 but declined. That year, the legendary Huey Pierce, Long, Jr., of nearby Winn Parish, first ran for governor but fell short of victory, losing to Henry L. Fuqua, the first Louisiana governor to die in office, In 1928, Long rebounded to win his single term as governor.
Johnson was a Presbyterian, a Mason and a Shriner. At the time of the publication of Henry E. Chambers's A History of Louisiana, Vol. II, in 1925, he was sixty-eight years of age.
A. R. Johnson is interred at Springville Cemetery in Coushatta in Red River Parish. Johnson's grandson, Andrew Pittman Johnson (1913–1993), lived in Homer. A Johnson great-grandson, Tony Johnson, is a real estate agent in Homer.