Born in Glasgow, Sermanni played as a midfielder in Scotland, England, Australia and New Zealand for Cumbernauld United, Albion Rovers, Blackpool, Torquay United, Dunfermline Athletic, Canberra City and Christchurch United.
Sermanni has coached a number of Australian club sides, including Canberra Metros and Canberra Cosmos.
In 2001 Sermanni was an assistant coach for the Bay Area CyberRays of the Women's United Soccer Association (WUSA). The CyberRays won the league's inaugural championship, the Founders Cup. He remained an assistant coach in 2002 when the team changed its name to the San Jose CyberRays. In 2003, he was hired as head coach of the New York Power (WUSA), who had fired their previous coach after finishing their 2002 season with a dismal record of 3 wins, 17 losses, and 1 draw (10 pts). With him as their coach, the team improved to finish the 2003 season in fifth place with a record of 7 wins, 9 losses and 5 draws (26 pts.).
Sermanni took up a job as coach of Australia women in December 2004, having previously also coached the team between 1994 and 1997. Among his accomplishments as coach of Australia women, are quarter-finals at 2007 and 2011 FIFA Women's World Cup.
On 30 October 2012, US Soccer Federation announced that starting 1 January 2013, Sermanni would be the head coach of the United States women's national soccer team, ending his long-time association with Australia.
Sermanni finished his first year as manager of United States unbeaten with 13 wins and 3 draws. The 3 draws are 1–1 against Sweden at the Algarve Cup, and 2 friendlies, 3–3 with Germany, and 1–1 with New Zealand. For the second time the United States women's national team posted an unbeaten record in a year that featured a double-digit number of matches. In addition, the team won the 2013 Algarve Cup.
The United States women started 2014 with friendlies, under Sermanni, a 1–0 win over Canada, and 7–0 and 8–0 drubbings of Russia. At the 2014 Algarve Cup, the United States women did not win a game in group stage: 1–1 with Japan, 0–1 loss to Sweden and a 3–5 loss to Denmark. The team finished seventh with a 3–0 win over Korea DPR, the lowest the team had finished at the Agarve Cup, which the team had won 9 times.
On 6 April 2014, Sermanni was relieved of his coaching duties following a 2–0 win over China in an international friendly.
In 2014 Sermanni joined Canada women's national soccer team's coaching staff as technical consultant. Sermanni was on contract as an assistant coach to Canada during 2015 FIFA Women's World Cup.
On 20 October 2015, Orlando City Soccer Club announced the creation of a women's team Orlando Pride to join National Women's Soccer League in 2016, and Sermanni would be the first coach for the expansion club.