Gray was born in 1826 in Limerick, Ireland. His father was George Blanchard Grey [sic], a captain in the British Army's 67th (South Hampshire) Regiment of Foot, who died whilst stationed in India with his regiment shortly before Andrew's birth. His mother was Mary Ann Sexton, who died during his childhood, with Gray being raised by his aunt.
Gray undertook medical training in Dublin—which at the time involved studying subjects taught at various hospitals around the city, based on the model of medical training in Edinburgh—completing it in around 1846. For a number of years after this, he was the assistant of William Wilde, a renowned eye and ear surgeon and father of the writer Oscar Wilde.
On 9 January 1852, Gray passed the examinations for admission to the Royal College of Surgeons of England.
Shortly after qualifying for RCS membership, Gray was appointed as a surgeon-superintendent for the Colonial Land and Emigration Commission, which involved accompanying migrants on assisted passage schemes as the ship's chief medical officer, as well as maintaining their welfare and discipline. Thus Gray would go on to make several voyages to colonial Australia around the time of the colonies' gold rushes.
On 3 November 1858, Gray married Elizabeth Ann McNalty (daughter of fellow surgeon George William McNalty) at St Peter's Church in Dublin.
In January 1859, Gray and his new wife embarked for the colony of Victoria, on board the emigrant ship Hornet. They spent over a month in Sydney, until they could obtain passage to Melbourne where they arrived on 2 May 1859. The colony of Victoria had no compulsory registration to practice medicine, although there was a voluntary register, to which Gray added his name on 2 May 1859.
Through a relative on his mother's side (George Sexton Evans), who was one of the contractors, Gray was employed as resident railway surgeon during the construction of the Geelong–Ballarat railway line. Following the completion of the line in March 1862, and the subsequent drop in the workforce, Gray's position was terminated and he moved to Melbourne late in the year. He began practice as a surgeon and ophthalmologist in Russell Street for several months, and then prompted by the number of patients seeking treatment for eye and ear disease, founded the Eye and Ear Infirmary in rented premises at 2 Albert Street, East Melbourne.
Gray died at his home at 13 Collins Street, Melbourne on 10 July 1907, aged 81 years. He had been suffering from cystitis, bronchopneumonia and exhaustion, but nonetheless had continued to work at the Eye and Ear Hospital until two weeks before his death.
Gray's wife, Beth, died on 19 July 1928, aged 80. They were survived by eleven of their fourteen children (one son and ten daughters).