Murray was born at Cobourg, Canada West on 3 March 1841. He was the second son of Elizabeth Macfarlane and Robert Murray, an accountant. His parents had emigrated from Scotland to Ontario in about 1834. He went to school in London, Ontario and later to Cobourg College. In 1858, at the age of seventeen he returned to Scotland to live with his grandfather, John Macfarlane, and continue his education at Stirling High School. In 1864 he enrolled at University of Edinburgh to study medicine however he did not complete his studies and did not graduate.
In 1868 he joined the whaling ship, Jan Mayen, as ship's surgeon and visited Spitsbergen and Jan Mayen Island. During the seven-month trip he collected marine specimens and recorded ocean currents, ice movements and the weather.
On his return to Edinburgh he re-entered the University to complete his studies (1868–72) in geology under Sir Archibald Geikie.
In 1872 Murray assisted in preparing scientific apparatus for the Challenger Expedition under the direction of the expedition's chief scientist, Charles Wyville Thomson. When a position on the expedition became available Murray joined the crew as a naturalist. During the four-year voyage he assisted in the research of the oceans including collecting marine samples, making and noting observations and making improvements to marine instrumentation. After the expedition Murray was appointed Chief Assistant at the Challenger offices in Edinburgh where he managed and organised the collection. After Thompson's death in 1882 he became Director of the office and published in 1896 The Report on the Scientific Results of the Voyage of HMS Challenger, a work of more than 50 volumes of reports.
Murray renamed his house, on Boswall Road in northern Edinburgh, Challenger Lodge in recognition of the expedition. The building now houses St Columba's Hospice.
In 1884, Murray set up the Marine Laboratory at Granton, Edinburgh, the first of its kind in the United Kingdom. In 1894, this laboratory was moved to Millport, Isle of Cumbrae, on the Firth of Clyde, and became the University Marine Biological Station, Millport, the forerunner of today's Scottish Association for Marine Science at Dunstaffnage, near Oban, Argyll and Bute.
After completing the Challenger Expedition reports, Murray began work surveying the fresh water lochs of Scotland. He was assisted by Frederick Pullar and over a period of 3 years they surveyed 15 lochs together. In 1901 Pullar drowned as a result of an ice skating accident which caused Murray to consider abandoning the survey work. However Pullar's father, Laurence Pullar, persuaded him to continue and gave £10,000 towards completion of the survey. Murray coordinated a team of nearly 50 people who took more than 60,000 individual depth soundings and recorded other physical characteristics of the 562 lochs. The resulting 6 volume Bathymetrical Survey of the Fresh-Water Lochs of Scotland was published in 1910. The cartographer John George Bartholomew, who strove to advance geographical and scientific understanding through his cartographic work, drafted and published all the maps of the Survey.
In 1909 Murray indicated to the International Council for the Exploration of the Sea that an oceanographic survey of the north Atlantic should be undertaken. After Murray agreed to pay all expenses, the Norwegian Government lent him the research ship Michael Sars and its scientific crew. He was joined on board by the Norwegian marine biologist Johan Hjort and the ship departed Plymouth in April 1910 for a four-month expedition to take physical and biological observations at all depths between Europe and North America. Murray and Hjort published their findings in The Depths of the Ocean in 1912 and it became a classic for marine naturalists and oceanographers.
He was the first to note the existence of the Mid-Atlantic Ridge and of oceanic trenches. He also noted the presence of deposits derived from the Saharan desert in deep ocean sediments and published a vast number of papers on his findings.Fellow of the Royal Society of Edinburgh (1877)
Neill Medal from the Royal Society of Edinburgh (1877)
Makdougall Brisbane Prize from the Royal Society of Edinburgh (1884)
Founder's Medal from the Royal Geographical Society (1895)
Fellow of the Royal Society (1896)
Knight Commander of the Order of the Bath (1898)
Cullum Geographical Medal from the American Geographical Society (1899)
Clarke Medal from the Royal Society of New South Wales (1900)
Livingstone Medal from the Royal Scottish Geographical Society (1910)
Vega Medal from the Swedish Society for Anthropology and Geography (1912)
Other awards included the Cuvier Prize and Medal from the Institut de France and the Humboldt Medal of the Gesellschaft für Erdkunde zu Berlin.
He was president of the Royal Scottish Geographical Society from 1898 to 1904.
The John Murray Laboratories at the University of Edinburgh, the John Murray Society at the University of Newcastle and the Scottish Environment Protection Agency research vessel, the S.V. Sir John Murray, and the Murray Glacier are named after him.
Cirrothauma murrayi, an almost blind octopus that lives at depths from 1,500 m (4,900 ft) to 4,500 m (14,800 ft) and the Murrayonida order of sea sponges are named after Murray.
In 1911, Murray founded the Alexander Agassiz Medal which is awarded by the National Academy of Sciences, in memory of his friend Alexander Agassiz (1835–1910).
After his death his estate funded the John Murray Travelling Studentship Fund and the 1933 John Murray Mabahiss Expedition to the Indian Ocean.
Murray was killed when his car overturned near his home on 16 March 1914 at Kirkliston near Edinburgh. He is buried in Dean Cemetery in Edinburgh on the central path of the north section in the original cemetery.