Smith was an alderman on Toronto City Council in the 1970s. He served a period as deputy mayor and president of city council. He ran for Mayor of Toronto in 1978, but was defeated by John Sewell in a three-way split. Smith became a backroom lobbyist for developers and was instrumental in helping Art Eggleton defeat Sewell in 1980.
After his defeat in municipal politics, Smith ran for and was elected to the Canadian House of Commons in the 1980 election as the Liberal Member of Parliament (MP) for the riding of Don Valley East in a suburb of Toronto.
In 1982, he became Deputy Government House Leader, and joined the Canadian Cabinet in 1983 as Minister of State for Small Businesses and Tourism. Smith was appointed to the same position when John Turner became Prime Minister of Canada in 1984. Smith was defeated in the subsequent 1984 election.
Smith returned to the legal profession and serves as Chairman Emeritus of Dentons LLP, one of Canada's largest law firms. In the 1990s, he worked as a senior backroom adviser to Liberal leader Jean Chrétien, playing a leading role in the party's election campaigns.
A Chrétien loyalist, Smith was appointed to the Senate in 2002, and was outspoken in his support for Chrétien against attempts by Paul Martin to force the Prime Minister to retire. Since Martin became Liberal leader, Smith has urged party unity.
He was a key backer of former Liberal leadership candidate Michael Ignatieff. Subsequently, he was appointed by Liberal Leader Stéphane Dion to serve as party's National Campaign Co-Chair with Mark Marissen and Nancy Girard.
On January 29, 2014, Liberal Party leader Justin Trudeau announced all Liberal Senators, including Smith, were removed from the Liberal caucus, and would continue sitting as Independents. The Senators referred to themselves as the Senate Liberal Caucus even though they were no longer members of the parliamentary Liberal caucus.
Smith retired from the Senate upon reaching the mandatory retirement age of 75 on May 16, 2016.
Smith also serves on the cabinet of the capital campaign of Tyndale University College and Seminary in Toronto.