Mahoney was educated at Richview Collegiate in Etobicoke, Toronto. From 1978 to 1987, he served as a councillor on the Mississauga City Council and the Regional Council of Peel.
Mahoney was first elected to the Ontario legislature in the 1987 provincial election, defeating his closest opponent by over 14,000 votes in the newly created riding of Mississauga West. For the next three years, he served as a backbench Member of Provincial Parliament (MPP) supporting the government of David Peterson.
The Liberals lost the 1990 provincial election, though Mahoney was re-elected without difficulty and became Chief Opposition Whip. He ran for the leadership of the Ontario Liberal Party in 1992, although he had only a minimal support base within the party. Most political observers believed he was attempting to increase his public profile, to be assured of a cabinet position when the Liberals returned to power. He was generally regarded as being on the right-wing of the party during this campaign.
Mahoney placed fifth out of six candidates on the first ballot of the convention. He withdrew from the contest after the second ballot, and endorsed Lyn McLeod. (This support was pivotal to McLeod's victory on the fifth ballot, as she defeated runner-up Murray Elston by only nine votes.)
The Progressive Conservative Party won a majority government in the 1995 election, and Mahoney was narrowly defeated in Mississauga West by PC candidate Rob Sampson. He switched to federal politics two years later, and was easily elected to the Canadian House of Commons for Mississauga West in the 1997 federal election, defeating his nearest opponent by over 21,000 votes. He scored an equally easy victory in the 2000 federal election.
Mahoney supported Paul Martin in the 1990 federal Liberal leadership convention, but subsequently opposed efforts by other Martin supporters to remove Jean Chrétien from the leadership position. During his time in parliament, he was considered a Chrétien loyalist. On April 11, 2003 Chrétien appointed Mahoney Secretary of State for selected Crown Corporations. His primary responsibility was the Canadian Mortgage and Housing Corporation, and he was generally regarded as minister responsible for affordable housing. Mahoney was demoted to the backbenches again when Paul Martin became Prime Minister of Canada on December 11, 2003.
As a result of redistribution, Mahoney was forced to run against fellow Liberal Member of Parliament Carolyn Parrish for the Liberal nomination in Mississauga—Erindale in the 2004 federal election. Parrish was often criticized in the Canadian media for making statements that were interpreted as anti-American, while Mahoney frequently condemned anti-American rhetoric during his time in the Commons. (On February 27, 2003, he ended one spoken-word intervention by yelling "God Bless America".)
Parrish defeated Mahoney for the nomination, 2,165 votes to 1,925. After losing the contest, Mahoney told reporters that Parrish was "so low in the gutter that it's a shame that she's standing to represent the Liberal party". Mahoney did not contest in the 2004 election.
In 2006, Mahoney was appointed Chair of the Workplace Safety and Insurance Board of Ontario. He announced his retirement in March 2012. Elizabeth Witmer has been appointed as his successor.
On March 17, 2014, Mahoney announced that his candidacy to replace the retiring long-serving Mayor of Mississauga, Hazel McCallion, in the 2014 municipal elections. He was placed second in the election, losing to former MP and City Councillor Bonnie Crombie.