A PT instructor in the Royal Artillery, Carmichael signed for Lincoln City on a month's trial in 1989. Thrust straight into Lincoln's first team at centre-forward, Carmichael responded by scoring, on his debut against Scunthorpe United. In his second season at the club, Carmichael made occasional appearances at centre-half as well as centre-forward and spent both the 1991–1992 and 1992–1993 seasons as a virtual ever present at the heart of Lincoln's defence. In 1991/92 he was Lincoln's Player of the Season. In the summer of 1993 he left Lincoln for Scunthorpe United, in a part-exchange deal with Dave Hill.
Restored to the forward line at Scunthorpe, Carmichael scored a total of 24 goals, with 18 in the league, during a highly successful 1993–1994 season. He also established a club record when between 28 December 1993 and 19 February 1994 he scored in eight successive matches. However, he struggled to maintain such form the following season and joined Barnet on a month's loan in September 1994 before joining Preston North End in March 1995 for the remainder of the season, briefly playing alongside a young David Beckham.
He made a solitary, goalscoring, appearance for Mansfield Town at the start of the 1995–1996 season before linking up with Doncaster Rovers and then, at the end of February, Darlington. He helped Darlington reach the 1996 Endsleigh League Division Three play-off final where they were defeated 1–0 by Plymouth Argyle on 25 May 1996 at Wembley Stadium.
Following his release by Darlington, Carmichael played four games for TPS Turku in the Finnish Veikkausliiga, and he scored one goal for them, on 30 June 1996 against HJK Helsinki. He then joined Aylesbury United, and he scored on his debut for them in the 2–1 away defeat to Hitchin Town on 17 August 1996. His last game for the Ducks was on 12 April 1997 against Heybridge Swifts. Carmichael then linked up with Ilkeston Town, under the management of the man who transferred him from Lincoln, Keith Alexander.
Carmichael later played for Boston United in the 1998/99 season (where he scored 20 times in 34 matches), and had a 10-game spell with Grantham Town in the 1999–2000 season.
He later joined Wisbech Town, then under the management of his former Boston United teammate Ian Stringfellow, debuting in the 1–1 home draw with Histon on 26 September 2000. However, after just five league games and one goal for the Fenman, financial constraints saw six of the squad, Carmichael included, depart the club following the 2–0 home defeat to Newport IOW on 21 October 2000. Carmichael moved on to Lincoln United but a persistent back injury curtailed his appearances and he wound his career down with various clubs in the Lincoln & District Sunday League.
After being kit manager at Lincoln City he was appointed in May 2012 as manager at Lincoln Moorlands Railway but he left the club after three months.