Savage was born in Hoxton, London, on 10 April 1779. He was educated at a private school in Stockwell and then articled to Daniel Asher Alexander, architect of the London Docks, for whom he worked for several years as clerk of the works. He became a student at the Royal Academy in 1796.
In 1800 he won second prize in a competition for a scheme of improvements to the city of Aberdeen and five years later came first in a competition to rebuild the Ormond Bridge over the Liffey in Dublin, which had been swept away by a storm. The project was delayed, and it was decided instead to build the new bridge about 50 metres west of the destroyed one. Savage exhibited his design for the new location as Richmond Bridge forming the approach to the Four Courts, Dublin at the Royal Academy in 1809. A three-arched bridge built of granite, with cast-iron balustrades, it was constructed in 1813–16. Originally named after the Duke of Richmond, it is now known as the O'Donovan Rossa Bridge. In 1815 Savage won a competition to design a river-crossing at Tempsford in Bedfordshire with another three arched bridge.
In 1819 his plans for the new parish church of St Luke, Chelsea were chosen from among more than 40 submissions. It was an ambitious building, costing £40,000 and designed to accommodate 2,500 people. He designed it in imitation of the Gothic churches of the 14th and 15th centuries, with solid stone vaulting supported by flying buttresses; Charles Locke Eastlake described it as "probably the only church of its time in which the main roof was groined throughout in stone". Savage originally intended the tower to have an open spire, like that of Wren's St Dunstan-in-the-East, but this was forbidden by the Board of Trade. Eastlake, writing in the 1870s, criticised the building for its "machine made look" and " the cold formality of its arrangement". Savage designed several other, less ambitious Gothic churches, and one, St James, Bermondsey, in a Classical style.
He submitted designs for the new London Bridge to a committee of the House of Commons in 1823. He told the committee that he had used the same principles in designing the arches that he had in the vaults of St Luke, Chelsea, where, he said, there had not been "the slightest settlement in any part of the building, nor even a thread opening in any of the joints of the courses to indicate any strain or inequality of pressure." The committee gave his design a positive reception, but chose one by John Rennie instead on the casting vote of the chairman.
In 1825 he drew up a plan which he called the "Surrey Quay" for embanking the south bank of the Thames, from London Bridge to Lambeth.
In 1830 Savage became architect to the Society of the Middle Temple for whom he built the Plowden Buildings, and a clock tower for their hall. In 1840 the society commissioned him to restore the Temple Church. The work was well underway when he was dismissed due to a disagreement with the building committee, leaving the work to be completed by Sydney Smirke and Decimus Burton. His other restoration work included repairs to the belfries of St-Mary-le-Bow, London, and the Broad Tower of Lincoln Cathedral, and considerable alterations to St Mary-at-Hill, London, where he worked in 1827-8, and again at the very end of his life.
In 1836, having unsuccessfully entered the competition to design the new Houses of Parliament, he published a pamphlet entitled Observations on Style in Architecture, with suggestions on the best mode of procuring Designs for Public Buildings and promoting the improvement of Architecture, in which he criticised the stipulation of a particular style for competition entries, and more generally attacked the slavish imitation of historical styles:
The architect and his patron are not aware that this piecemeal copying of details is quite compatible with an entire ignorance and neglect of all the more essential qualities for which the antique examples have been admired. They owe their effect to their singleness of intention, simplicity of means, beauty of proportion, and the all-pervading harmony of the totality, to which the details are most profoundly subordinate; for the perfection of the work is, when the parts are nothing and the totality everything. The end is felt, not the means.
Much of Savage's practice involved arbitration cases and the investigation of architectural and engineering questions in court. Among these was the protracted Custom House case of the Crown v. Peto, in which the defendant Henry Peto attributed his success mainly to Savage's evidence.
He was a member of the Surveyors' Club, and, for many years, member and chairman of the Committee of Fine Arts of the Society for the Promotion of Arts, Manufactures, and Commerce. He was a founder member of the Graphic Society, a member of the Institution of Civil Engineers, a member of the Architectural Society, and, briefly a fellow of the Institute of British Architects, from which he resigned after a disagreement. He exhibited at the Royal Academy from 1799 to 1832.
He died on 7 May 1852 and was buried in St, Luke's, Chelsea.Richmond Bridge, Dublin (1808).
Church of St John of Jerusalem, Well Street, Hackney (1810). Demolished.
Tempsford Bridge, Bedfordshire (1815).
St Luke's Church, Chelsea (1820–24).
Clock tower and Plowden-buildings, Middle Temple.
St. James' Church, Bermondsey (1827–29).
Trinity Church, Sloane Street (1828–30). Demolished.
Holy Trinity Church, Tottenham Green (1828–29).
St. Mary's Church, Ilford, Essex (1829–31).
St. Mary's Church, Speenhamland, near Newbury, Berkshire (1829–31). Demolished.
Bull-and-Mouth-Inn, also known as the Queen's Hotel, St. Martin's-le-Grand, London (1831). Demolished.
St. Michael's Church, Burleigh Street, Strand, London (1832–33). Demolished.
St. Thomas the Martyr Church, Brentwood, Essex (1835). Demolished and replaced.
Rectory of St Mary-at-Hill, City of London (1834).
St. Paul's Church, Addlestone, Chertsey, Surrey.
All Saints' Church, Beulah Hill (1837).
Tenterden Union Workhouse, Kent (1843). Demolished.
Bromley Union Workhouse, Locks Bottom, Kent (1844). Designed in collaboration with SO Foden. Demolished.
Two bridges at Reading, Berkshire.
Baptists College, Stepney.
An Essay on Bridge Building in the Transactions of the London Architectural Society.
Observations on the Varieties of Architecture, used in the structure of Parish Churches 1812).
Prospectus of a Plan for the Surrey Quay (1825).
Observations on the Proposed New London Bridge (1828).
Observations on Style in Architecture, with suggestions on the best mode of procuring Designs for Public Buildings and promoting the improvement of Architecture; especially in reference to a recommendation in the Report of the Commissioners on the Designs for the New Houses of Parliament (1836).