Abernethy was born in Aberdeen to George Abernethy and Isabella Johnston. In 1823, the family moved to South Wales, where his father managed the Dowlais Ironworks, and in 1826 moved to Southwark in London, because his father obtained a job as a foundry manager. While there, he watched the construction of London Bridge. In 1827, he was sent with his brothers to Cotherstone Boarding School in the North Riding of Yorkshire, but was removed by his uncle, Revd. John Abernethy, two years later, when he discovered the awful conditions at the school. His uncle took him to London and then to Haddington, East Lothian where he spent two years at the Grammar School. He then went to work under his father, who was working on the construction of the Eastern Dock, which was part of the London Docks complex.
His next move with his father was in 1832 to Herne Bay, where a timber pier was being constructed. However, he sailed to Sweden in 1833, to lay out new roads for a manganese mine near Jönköping which a friend had bought. He spent much of his spare time sketching the architecture and scenery of the area, until he was recalled to England in 1835 by his father, to assist on the Start Point lighthouse project in Devon. This appears to be the last time he worked with his father, as he moved to Goole in 1836, to work with George Leather on the construction of the steamship dock and lock which connected the Aire and Calder Navigation to the Humber. Despite nearly drowning when a cofferdam collapsed, he devoted most of the rest of his career to marine engineering. He worked on improvements to the Aire and Calder between Wakefield and Methley until 1838, and then became the resident engineer for the North Midland Railway, working under George Stephenson. However, he resigned after 18 months, to become the engineer for the Aberdeen Harbour Trust.
Aberdeen had a tidal harbour at the time Abernethy arrived, and he spent a year dredging and building embankments to improve the access channel. In the following year, a competition was held for the design of an enclosed dock, and his design was selected. An Act of Parliament was obtained to implement the design, but Abernethy first had to convince an independent assessor in London of its soundness. The meeting was inconclusive, but the chairman of the Harbour Trustees was satisfied, and tenders were sought for construction. The lowest tenderer was given the contract, but could not complete it, and Abernethy took over after a year, using direct labour to finish the work. When it was built, the entrance lock was the largest in Britain, measuring 250 by 60 feet (76 by 18 m), with a navigable depth of 22 feet (6.7 m) at high water.
The Preliminary Enquiries Act was passed to ensure that major new schemes were competently assessed before being implemented, and Abernethy worked as one of its Surveying Officers for eight years until 1852. During this time he held public enquiries into schemes for improvements to the River Clyde, the River Tyne and the River Ribble, and for the construction of docks at Belfast, Birkenhead, Glasgow, Liverpool and Newcastle upon Tyne. In addition to obtaining skills in the conduct of such meetings, he met with the leading engineers of the time, and learned the best practices for marine engineering. He acted as a consultant to the Swansea Harbour Trustees from 1847, and became their Engineer-in-chief in 1849, but continued to live in Aberdeen until 1851, when he moved to Birkenhead, where he held the same post as well.
He produced plans for Birkenhead Docks, as he was not convinced that the rival plans, produced by James Rendel, were practicable. Both plans were assessed by Admiral Sir Francis Beaufort and Robert Stephenson, who found in favour of Rendel. Following a takeover by Liverpool Corporation in 1855, Rendel's plans were implemented, but the sluices failed soon after the docks were completed in 1864. A new Act of Parliament obtained in 1866 authorised the rebuilding of the docks to Abernethy's plans. Throughout this period he was active with other schemes as well, including a shipyard for Messrs. Laird on the River Mersey. He produced plans for improvements to the Irish Bann Navigation in 1851, and for the Dean Forest, Monmouth, Usk and Pontypool Railway in the following two years.
He set up an office in London in 1853, and acted as consulting engineer for a large number of schemes, while still maintaining his regular oversight of the docks at Cardiff, Fraserburgh, Newport and Swansea. He expanded his work to include overseas projects in 1862, although a number of his recommendations were not implemented, and the scheme that was eventually built at Alexandria harbour was rather less satisfactory than his own design. Major harbour schemes included those at Silloth, Portpatrick, Falmouth, Port Natal in South Africa, Watchet, Boston and the Alexandra Dock at Kingston upon Hull. He also worked on the Swansea and Neath Railway between 1862 and 1863, the Turin and Savona Railway in Italy, which was 120 miles (190 km) long and included a 4-mile (6.4 km) tunnel, and the Hayling Island Railway. Between 1862 and 1867, he was responsible for the Grand Canal Cavour, a 54-mile (87 km) irrigation canal, which entailed him visiting Italy every four months. He used the opportunity this presented to visit Venice several times. In 1883, he reported on the three rival schemes for the Manchester Ship Canal, finding in favour of the one by Sir Edward Leader Williams. He acted as consulting engineer, visiting the site every month between 1885 and 1893, while Williams was Engineer-in-chief. His last major scheme overseas was the reclamation of Lake Aboukir in Egypt, between 1888 and 1889, although his regular dock work in Britain continued until his death, and work on the Bute Dock at Cardiff was still in progress when he died. It was completed later that year.
He became a member of the Institution of Civil Engineers in 1844, and was its president between December 1880 and December 1881. Although he only presented a single paper to it, he contributed to the discussions on a wide range of subjects, He was also made a fellow of the Royal Society of Edinburgh, and sat on two Royal Commissions. The first considered Metropolitan Sewage Discharge, and was held in 1882, while the second was held in 1889, and looked at Irish Public Works.
He married Ann Neill in 1838, and they had seven children, four sons and three daughters. Three of the sons worked with their father, and he formed a partnership with two of them, James and George, in 1893. He died at Broadstairs in Kent on 8 March 1896, his first son James taking over the engineering practice and his second son John compiling his biography in 1897.
Abernethy's cousin, also called James Abernethy (1809-1879), was an iron founder based in Aberdeen, who was responsible for the production of a number of bridges in Scotland. He also had a son called James, and the business continued until the middle of the twentieth century.