Born in Meaford, Canada West (now Ontario) to a Congregationalist minister, Duff received a Bachelor of Arts in mathematics and metaphysics from the University of Toronto in 1887. After graduation, he taught at Barrie Collegiate Institute while studying for the bar. Duff later took courses at Osgoode Hall Law School and was called to the Ontario Bar in 1893.
Duff practiced as a lawyer in Fergus, Ontario after being called to the bar. In 1895, Duff moved to Victoria, British Columbia and continued his career there. In 1895, he was made a Queen's Counsel. In 1903, he took part, as junior counsel for Canada, in the Alaska Boundary arbitration.
In 1904, he was appointed a puisne judge of the Supreme Court of British Columbia. In 1906 was appointed a justice of the Supreme Court of Canada. On January 14, 1914, he was appointed to Her Majesty's Most Honourable Privy Council. Duff was the first and only Puisne Justice of the Supreme Court of Canada to be appointed to the Imperial Privy Council. In 1924 he was elected honorary bencher of Gray's Inn, at the recommendation of Lord Birkenhead.
In 1931, he served as Administrator of the Government of Canada between the departure of Lord Bessborough for England and the arrival of Lord Tweedsmuir. Duff took on the position, as the Chief Justice was unavailable. As Administrator, Duff opened Parliament and read the Speech from the Throne on 12 March 1931, becoming the first Canadian-born person to do so.
In 1933, Duff was appointed Chief Justice of Canada, succeeding to Chief Justice Anglin. He was made a Knight Grand Cross of the Order of St. Michael and St. George the following year as a result of Prime Minister Richard Bennett's temporary suspension of the Nickle Resolution.
When Governor General Lord Tweedsmuir died in office on February 11, 1940, Chief Justice Duff became the Administrator of the Government. He held the office for nearly four months, until King George VI appointed Alexander Cambridge, 1st Earl of Athlone as Governor General on June 21, 1940. Duff was the first Canadian to hold the position, even in the interim. A Canadian-born Governor General was not appointed until Vincent Massey in 1952.
Duff also heard more than eighty appeals on the Judicial Committee of the Privy Council, mostly Canadian appeals; however, he never heard Privy Council appeals from the Supreme Court of Canada while he served on the latter. The last Privy Council appeal heard by Duff was the 1946 Reference Re Persons of Japanese Race.
In 1942, Duff served as the sole member of a Royal Commission constituted to examine the Liberal government's conduct in relation to the defence of Hong Kong. The resulting report, which completely exonerated the government, proved to be controversial, and was seen by many as a whitewash.
Upon reaching the mandatory retirement age for judges in 1939, his term of office by three years by special Act of Parliament; in 1943, his term of office was extended for another year by Parliament. He retired as Chief Justice in 1944.
Duff employed a conservative form of statutory interpretation. In a 1935 Supreme Court judgment, he detailed how judges should interpret statutes:
The judicial function in considering and applying statutes is one of interpretation and interpretation alone. The duty of the court in every case is loyally to endeavour to ascertain the intention of the legislature; and to ascertain that intention by reading and interpreting the language which the legislature itself has selected for the purpose of expressing it.
Duff has been called a "master of trenchant and incisive English," who "wrote his opinions in a style which bears comparison with Holmes or Birkenhead." A former assistant of Duff, Kenneth Campbell, argued that Duff was "frequently ranked as the equal of Justices Holmes and Brandeis of the United States Supreme Court," and Gerald Le Dain asserted that Duff "is generally considered to have been one of Canada's greatest judges." Other writers have taken a less favourable view, instead arguing that Duff's reputation is largely unearned; his biographer concluded that he was not an original thinker, but essentially a "talent student and exponent of the law rather than a creator of it."
More recent commentary has focused on Duff's legal formalism and its effect on Canadian federalism. Bora Laskin attacked Duff's decisions, arguing that Duff used circular reasoning and hid his policy-laden decisions behind the doctrine of stare decisis. As well, Lionel Schipper noted that, in reviewing Duff's judgments, it was:
apparent that he has given certain factors very little consideration in formulating his decisions. ... In constitutional cases, not only are the actual facts of the case significant but the surrounding social, economic and political facts are equally significant. A shift in these latter factors is as important in deciding a case as any other change in the facts. It is this consideration that Chief Justice Duff ignored.