A solicitor, businessman and citizen soldier before his entry into politics, Gould was a member of the New South Wales Legislative Assembly from 1882 to 1898, during which time he served as Minister for Justice in two Free Trade governments. He later served two years in the New South Wales Legislative Council from 1899 to 1901 until his election to the Australian Senate. Gould's interest in parliamentary procedure saw him become involved with the relevant standing committee and he was elected unopposed as the second President of the Senate in 1907. His tenure is remembered as more traditionalist and Anglophilic than his predecessor's.
Defeated by the Labor nominee in 1910 following the Liberal government's defeat, Gould remained in parliament as a backbencher until 1917, when he retired after he was not re-endorsed by the Nationalist Party. He was active in community and religious affairs during his long retirement.
Gould was born in Sydney, the son of solicitor John Morton Gould and his wife Anne (née Livingstone). He attended William Woolls' school in Parramatta, and went on to study law at the University of Sydney, although he did not take a degree. He served his articles with his father and was admitted to the bar in December 1870. He then worked in Singleton for a Sydney legal firm. Gould also developed significant business interests, being involved with the Great Cobar Copper Mining syndicate and serving as a director of the Electric Light and Power Supply Company, the City Bank of Sydney and the Oriental Timber Corporation. On 12 September 1872, he married Jeanette Jessie Maitland at St Paul's Church of England in West Maitland.
Gould was also a citizen soldier, enlisting as a volunteer in the West Maitland company of the New South Wales volunteer forces. He later took command of the Singleton Company, becoming a major in 1886. He later received the Volunteer Officers' Decoration for long service; he would retire from the regiment in 1902 as a lieutenant-colonel.
In 1882, Gould was elected to the New South Wales Legislative Assembly as the member for Patrick's Plains (renamed Singleton in 1894). Initially an opponent of the coalition associated with Henry Parkes and John Robertson, he later supported Robertson's government in 1885. By 1887 he was supporting his career with his own legal practice in Singleton and Sydney, and when the party system came into being in New South Wales in that year he identified as a Free Trader. In 1889 he was appointed Minister for Justice under Parkes, serving until 1891; he held the position again under George Reid from 1894 to 1898. During the first period of his ministry he became involved in a dispute with the Chief Justice, Sir Frederick Darley, over requests for improved court accommodation and a contested punishment for contempt of two witnesses. Despite embarrassing his premier, he remained close with Parkes throughout his state career and in later years frequently gave speeches in his memory.
Gould's career as justice minister was marked by consolidation of the law, tightening of licensing laws and reforms relating to police courts. A supporter of Federation, he nevertheless opposed the 1898 bill, which he believed deprived New South Wales of adequate recognition. He was defeated in the election of that year, but in 1899 he was one of Premier Reid's twelve appointments to the Legislative Council that enabled the passage of the legislation for a referendum on Federation.
Following Federation, Gould contested the first federal election in March 1901, standing for the Senate as a Free Trader. He was elected in the third of six positions, entitling him to a six-year term. His first speech was largely devoted to his impassioned support for free trade, and in his early years as a senator he also supported decentralisation and opposed proposals to establish a federal capital. A supporter of the White Australia policy, he expressed concern over Kanaka labour in Queensland, although his assumption that the arrangement was temporary enabled his pragmatism on this issue. Gould was also involved in defence matters, supporting the introduction of conscription and maintaining loyalty to the Empire, where "there are men much more experienced in the principles of government than we are". At the 1906 election, Gould was easily re-elected in the first position, winning the largest vote for any senator to that time.
From 1901 Gould maintained an interest in the procedure of the Senate, and was a member of the standing orders committee from its appointment in 1901 until 1907. He had supported the opening of Senate proceedings with prayer and had made frequent reference to the standing orders of the British House of Commons. He contested the Presidency of the Senate in 1904, but was defeated by the incumbent, Sir Richard Baker.
On Baker's retirement, Gould was unanimously elected President on 20 February 1907. He endeavoured to keep the Senate representative of the states' interests and free from party politics, and his rulings focused on unparliamentary language and relevance. Gould, appointed Knight Bachelor in 1908, differed from his predecessor in his adherence to British influence. With the election of the Fisher Labor Government in 1910, Gould was defeated by Harry Turley for the presidency.
Despite his support for conscription, Gould was not endorsed by the new Nationalist Party to contest the 1917 election. Deeply offended, Gould nevertheless elected to retire rather than run as an independent and split the Nationalist vote.
In his retirement, Gould continued his community involvement. A director of the Royal Prince Alfred Hospital and the Royal Alexandra Hospital for Children, he also served as a member and chancellor of the Sydney and Newcastle synods of the Church of England. He died in July 1936 at Rose Bay aged 89; he was survived by two sons and three daughters (his wife died in 1928; one daughter had also predeceased him). Gould was given a state funeral at St Andrew's Cathedral and was buried at South Head Cemetery.