Richmond was born in London, England, the son of Christopher Richmond, barrister and his wife, Maria Wilson. He was educated at Hackney Grammar School, at Hove House, Brighton and at the school attached to University College London. He was apprenticed to the engineer Samuel Clegg and from 1845 served on the staff of Isambard Kingdom Brunel for three years working on the Great Western Railway in southern England.
Richmond emigrated to New Zealand with his younger brother Henry Richmond on the Victory on 3 October 1850. The ship arrived in Auckland in February 1851 and the two walked south to Taranaki where they purchased a few acres near the home of their aunt Helen, who had married John Hursthouse and had also settled in Taranaki. Eventually members of the Richmond, Hursthouse, Atkinson and Ronald families, who were related by marriage, all settled near one another in the area.
Richmond returned to England in 1854 and married Mary Smith on 21 August 1856 before returning to New Zealand on the Kenilworth, which arrived in New Plymouth on 8 July 1857.
Richmond was elected unopposed as member of parliament for Omata at a 16 April 1860 by-election. He remained in Parliament until he resigned in 1865, as he was called to the Legislative Council, where he remained for only four months. He then represented Grey and Bell from 1866 to 1870, when he was defeated.
Mary had left for Nelson with other Taranaki refugees from the New Zealand Wars in 1860. In 1862 he joined her and became the editor of the Nelson Examiner while continuing his political career. After the fall of the Fox Ministry, he also became the Commissioner of Crown Lands. He served on the Nelson Provincial Council and was also appointed provincial secretary from 1863 to 1865.
He formed a close and lifelong friendship with John Gully and continued to paint and sketch in what little spare time he had.
Mary died in Nelson on 29 October 1865 having never fully recovered from the birth of her fifth child, and this event left Richmond 'harassed & broken'. However, by 1866 he was back in politics and moved his family to Taranaki. By 1869 the family had moved back to Nelson. Other family connections were also living there, including his brother William's family and his sister Maria and her husband, Arthur Atkinson.
Richmond travelled with his three eldest children to England and Europe in 1873 but returned to Nelson by January 1881. He continued to travel frequently. His daughter Dorothy Kate Richmond was an artist and art teacher.
Richmond died at the house of his daughter, Ann Elizabeth, in Otaki, which he was visiting, on 19 January 1898.