He was the eldest son of John Gregory (1724–1773) and Elizabeth Forbes (died 1761), and was born in Aberdeen. He was educated at Aberdeen Grammar School, King's College, University of Aberdeen, the University of Edinburgh (MD 1774), the University of Oxford, and Leyden University.
He accompanied his family moving to Edinburgh in 1764, and after going through the usual course of literary studies at that university, he was for a short time a student at Christ Church, Oxford. It was there probably that he acquired that taste for classical learning which afterwards distinguished him. He studied medicine at Edinburgh, and, after graduating doctor of medicine in 1774, spent the greater part of the next two years in Leiden, Paris, and in Italy.
Shortly after his return to Scotland he was appointed in 1776 to the chair his father had formerly held, and in the following year he also entered on the duties of teacher of clinical medicine in the Edinburgh Royal Infirmary.
In 1783 Gregory was one of the founders of the Royal Society of Edinburgh.
On the illness of William Cullen in 1790 he was appointed joint-professor of the practice of medicine, and he became the head of the School of Medicine at the University of Edinburgh on the death of Dr. Cullen in the same year.
As a medical practitioner Gregory was for the last ten years of his life at the head of the profession in Scotland (for part of which time he was in partnership with Thomas Brown, M.D.).
He was President of the Royal College of Physicians of Edinburgh from 1798 to 1801. His indiscretion in publishing certain private proceedings of the college led to suspension of his fellowship on the 13 May 1809.
Besides his Conspectus medicinae theoreticae, published in 1788 as a text-book for his lectures on the institutes, Gregory was the author of "A Theory of the Moods of Verbs", published in the Edin. Phil. Trans. (1787), and of Literary and Philosophical Essays, published in two volumes in 1792.
He married twice. His first wife, Mary Ross, died in 1784. They had no children.
He had a daughter named Sarah Gregory born in 1784.
In 1796 he married Isabella Macleod (1772–1847). They had eleven children: John Gregory (1797–1869), an advocate; Hugh Gregory (1799–1811); James Crawford Gregory (1801–1832), a doctor; William Gregory (1803–1858), a doctor and his twin Donald Gregory (1803–1836), an antiquary; Jane Macleod (1805–1813); Elizabeth Forbes (1808–1811); Margaret Craufurd/Crawford (1809–1849), who married her first cousin, William Pulteney Alison (1790–1859); Georgina (1811–1877); Duncan Farquharson Gregory (1813–1844), a mathematician; and Isabella (1816–1818).
The family lived together in their father's huge Georgian townhouse at 10 Ainslie Place on the Moray Estate in the eastern New Town of Edinburgh. James had bought the property as a brand new house shortly before his death.
His second wife's sister, Mary Macleod, was married to Sir George Mackenzie, 7th Baronet.
Gregory died after being run over by a horse and carriage in St Andrew Square in Edinburgh on 2 April 1821 and was buried in Canongate Kirkyard on the Royal Mile. His wife and most of his children lie with him. The grave lies in the extreme south-west corner, immediately to the right hand side of Adam Smith's grave.