Born to a poor family, Sharma took a B.A. from Lucknow University in 1925, followed by an M.A. He was awarded a Victoria gold medal from the university for his knowledge of Sanskrit. In 1931, he joined the Indian Judicial Service as a munsif, eventually becoming a judge on the Allahabad High Court, from which he retired in 1966.
His son Ramendra was a rear admiral in the Indian Navy. Justice Sharma moved to Gurgaon in 2005, where he died a month after his 111th birthday.