He was born at Bourne, Lincolnshire, in 1787. After his first twelve years of childhood spent at Bourne and Thirlby, a nearby village, and got his basic learning from village school-masters, he moved to Boston at the age of thirteen. Having been accoustomed with his parents to attend on the ministry of Binns, Baptist Minister at Bourne, he was influenced to attend on the ministry of D. Taylor, a Pastor of the General Baptist Church at Boston; later, was baptised into Christian fellowship, joined the Church, and began preaching occasionally at Swineshead in 1808. From 1809, with superintendence of the affairs of church experience at Boston, he moved to Sutterton in 1811 to work in the ministry as an assistant minister, and after three years at Sutterton he became Minister of the Baptist Church at Goverto, a village nearby to Sutterton.
In 1818, he moved to Grand Yarmouth in Norfolk and offered himself to the General Baptist Missionary Society, for missionary service in 1820. With delay in prior designed plans to travel India for missionary service along with William Ward, one of the Serampore Trio, he moved to London in 1820 and attended various courses of lecture, and hospital practice. He was ordained on 15 May 1821 at Loughborough, and sailed to India on 29 May 1821 along with William Ward, James Peggs, fellow-missionary and student at the General Baptist Academy at Wisbeach, and their wives.
He arrived at Madras(present Chennai on 25 September 1821. From Madras, they travelled to Serampore by 15 November 1821. He along with his fellow missionary James Peggs finally arrived at mission station at Cuttack via Calcutta(present Kolkata) by 12 February 1822. By 1822, Cuttack became a centre of missionary labour with an outstation at Puri—It was largely due to efforts of zealous Christian Claudius Buchanan, who strongly advocated for Christian institution near the temple Juggernaut(colloquial English name for temple Jagannath Temple, Puri - Ratha Yatra temple car), when he visited in 1806 - After his visit to Orissa, he created public opinion in England for sending missionaries to Orissa - By 1812, missionaries started making appeals to British Government for permission - With change in British government policy by 1813, they allowed the missionaries to work in Eastern India, officially - in 1816, New Connexion of General Baptists as a body resolved to do something to enlighten and evangelize the heathens with knowledge of Christianity.
He along with James Peggs, fellow-laborer, were deputed and directed to preach the gospel, superintending native schools, and acquiring the language of Orissa. In September 1823, he and his wife left Cuttack to form a new station at the temple of Jagannath Temple, Puri. On 17 September 1825, Bumpton wrote a letter that speaks the kind of work, he was engaged in day-to-day affairs:
He also says "On the whole, i never was so happy in the ministry before, and on the whole, i never was so much given up to it." [sic] On 31 October 1826, he writes again:
In 1825, Bampton metamorphed himself in native dress, but been criticized by other missionaries, and wrote to his friend as
Most of the times, his gospel message experienced, an utter rejection. In his own words, addressing his friend in England on 28 March 1827:
Between 1828 and 1829, he and his wife spent most of their time in Calcutta, mostly due to ill-health; returned to Puri in autumn of 1829, as his health didn't improve[due to diarrhoea], he died on 17 December 1830 at Puri.