Born on 11 September 1859 in Staines, Middlesex, he was the second son of William Smith, solicitor, and his wife, Harriet, daughter of Frederic Harcourt, of Ipswich. He attended Winchester College (1873–78) as a scholar. In 1879 He joined the department of Roman and Greek antiquities in the British Museum. He soon became known as a rising archaeologist, and in 1887 was a founder editor and contributor to the Classical Review.
In 1887 he was attached to the diplomatic mission in Persia. In 1892 he married Alice Edith, daughter of H. W. Watson of Burnopfield, Co. Durham. They had two sons, Simon and Gilbert.
From 1895 to 1897 he was granted special leave to take up the directorship of the British School in Athens. The school had just received an annual grant from the Treasury and was able to extend its activities. Harcourt-Smith enhanced the prestige of the school and instituted its "Annual". He also began the school's excavations in the island of Melos, which contributed much to the knowledge of Aegean civilisations. While in Athens, Harcourt-Smith was promoted to assistant keeper of his department at the British Museum.
In 1908 Harcourt-Smith became chairman of a commission of the Victoria and Albert Museum, set up to look into the collections of applied art at South Kensington that had been purchased by the government after the Great Exhibition of 1851. His report was so highly approved of that he was offered the post of director and secretary under the new organization, and in 1909 he accepted the appointment. The newly completed building gave scope for a more orderly display of the collections; the administration of the museum and its staff was set upon more modern lines, and the objects were grouped according to their material rather than, as in many great museums, by nationality or period. This method of grouping objects lasted until the evacuation of 1939 when they were grouped chronologically by Sir Leigh Ashton.
Harcourt-Smith remained at the Victoria and Albert Museum until his retirement in 1924. During this time he introduced many improvements. He raised the status of the technical staff and negotiated for them the same rank and pay as the officials of the British Museum. He added students' rooms to all departments, and a steady stream of catalogues and guides was begun. Official guide lecturers were instituted and sponsored special exhibitions such as the Franco-British Exhibition of 1921 were introduced. It was under his directorship that the museum acquired the Salting collection, the Rodin sculptures (now transferred to the Tate Gallery), the Talbot Hughes collection of costumes, the Alma Tadema library, the Le Blon Korean pottery and the Pierpoint Morgan stained glass.
A year after his retirement Harcourt-Smith was appointed advisor for the Royal Art Collections and from 1928 until 1936 he was also Surveyor of the Royal Works of Art. He played a leading part in the foundation of the Central Committee for the Care of Churches, he was chairman of the committee of the Incorporated Church Building Society, and vice-chairman of the British Institute of Industrial Art and the British Society of Master Glass Painters. He was also vice-president of the Hellenic Society, president of the Society of Civil Servants and British representative on the International Office of Museums. He was an honorary member of the British Drama League and an honorary associate of the Royal Institute of British Architects.
As well as contributing to many of the British Museum departmental catalogues, he wrote for the art journals and also published a number of monographs: The Collection of J. Pierpont Morgan (1913), The Art Treasures of the Nation (1929), and The Society of the Dilettanti: its Regalia and Pictures (1932).
Smith was knighted in 1909, appointed CVO in 1917, and advanced to KCVO in 1934. He died on 27 March 1944.