Born on 4 February 1813 at Penylan, near Llandovery, Carmarthenshire, he was the son of Rhys Jones, a small farmer and local preacher, and later independent minister at Ffald-y-brenin in the same county. He received his early education at Neuaddlwyd grammar school, at Rhydybont, Blackburn Academy, and at the Presbyterian College, Carmarthen.
Jones settled as minister of the independent church at Kilsby in Northamptonshire in January 1840, and was fully ordained there on 18 June of the same year. At this period he assumed the additional name "Kilsby". About 1850 he moved to Birmingham, and subsequently to Bolton. He returned to Wales, and bought Gellifelen farm, near Llanwrtyd, Brecknockshire, his mother's birthplace, where he built a house called Glenview. Except for a short period spent in London as pastor of the Tonbridge congregational chapel, he passed the rest of his life at Glenview.
Jones filled pulpits at Rhayadr (1857–60) and at Llandrindod Wells (1868–1889), where he built Christ Church Chapel, but did no ordinary pastoral work. He died on 10 April 1889, and was buried in the parish churchyard at Llanwrtyd, where a monument was placed over his grave by public subscription.
Jones preached in Welsh or English, had a reputation as a lecturer, and employed a biting wit. His best-known subjects were "Vicar Prichard", John Penry, and "Self-made Men". He was a resolute enemy of the church establishment in Wales, and a Liberal.
Jones translated into English William Rees's Memoirs of the late W. Williams of Wern, London, 1846; and into Welsh The second Letter on the present Defective State of Education in Wales, by W. Williams, M.P. for Lambeth, with a sketch of the educational policy of the government, Llanelly, 1848, and John Brown's Biblical Dictionary as Geiriadur Beiblaidd, Glasgow, 1869–70. He edited:Holl Weithiau prydyddawl a rhyddieithol … W. Williams, o Bantycelyn (The Complete Works of Williams of Pantycelyn, with Memoir), Glasgow, 1868;
a Welsh version of John Bunyan's Pilgrim's Progress and other works, Glasgow, 1869;
a Welsh Family Bible (new edition of Peter William's Bible), Glasgow, 1869.
He published A Lecture on the Educational Wants of Wales, 1851; and "An Essay on the Characteristics of Welsh Preaching" was included in Echoes from the Welsh Hills, by the Rev. David Davies, London, 1883. Jones was also joint author with Dr. R. Richardson of Rhayadr of Breconshire and Radnorshire Mineral Springs, Llanidloes, 1860.
Jones contributed to Welsh periodicals, beginning while at Kilsby with articles on political, social, and educational questions in Y Traethodydd and Y Byd Cymreig. For many years he was Welsh editor to William Mackenzie of Glasgow.
On 22 April 1842 Jones married Ann Southwall Chilcott of Leominster, who survived him. They had one son, Ryse Valentine Chilcott.