Rofe attended Whitechapel Secondary School, together with Terry Brisley, who later played alongside him at Leyton Orient. He also represented East London Schools.
He started his career at Leyton Orient, originally as an associate schoolboy in March 1964, turning professional in February 1968.
He was a fast, tough-tackling left-back who starred in Orient's 1969–70 Third Division title triumph. He was ever-present in 1970–71 and a key figure in Orient's 1971–72 FA Cup run. He netted six goals in 172 League appearances for Orient before following Jimmy Bloomfield to Leicester City for £112,000 in August 1972, which made him the most expensive full-back in British football at that time.
Rofe was signed by Bloomfield to fill the left back position after David Nish had been sold to Derby County for a record fee of £250,000. Rofe was an attacking full back with great pace and he made an instant impact in the Leicester City team. He remained faithful to Leicester after Bloomfield left the club in 1977, but was surprisingly sold to Second Division Chelsea, in February 1980, shortly before Leicester won promotion back to the First Division.
Rofe made his England U.23 debut as a City player in 1973. In his eight seasons at Leicester he played 290 league games for the club and scored six goals.
Rofe then spent three seasons at Chelsea as they drifted in Division 2, including a period as team captain.
In July 1982, Lawrie McMenemy signed him on a free transfer for First Division Southampton, where he played out the last two years of his playing career.
In 1984, he joined Southampton's coaching staff under the manager Chris Nicholl, initially as the reserve team coach, moving up to first team coach in 1987.
He joined Bristol Rovers as a coach in July 1991. Shortly after joining Rovers, he replaced Martin Dobson, firstly as caretaker manager, taking the position permanently in October 1991. Rovers finished the 1991–92 season in 13th position in Division 2. In November 1992, Rofe resigned.
He then moved briefly to Stoke City as the reserve team manager before returning to Southampton as the youth team coach. This position lasted until July 1997 when he moved on to Fulham as a coach, before a brief spell coaching for Kingstonian.
In April 1998, he returned to Southampton as coach of their Academy, but soon moved up to reserve team coach and, in March 2001, to first team coach, under Stuart Gray.
He served as first team coach and assistant manager as Southampton had a rapid turnover of managers. His greatest moment as a Southampton coach was when Southampton reached the 2003 F.A. Cup final under Gordon Strachan, losing 1–0 to Arsenal.
As assistant manager at Southampton, he had a fiery reputation. On 10 January 2002, he had to be separated from the acting Liverpool manager Phil Thompson by the fourth official in a touchline argument over a bad tackle on Brett Ormerod.
On 8 May 2004, Rofe was ordered from the technical area after protesting about the award of a penalty to Aston Villa.
Most infamous was probably the row with the Blackburn Rovers boss, Graeme Souness, on 25 October 2003, following an unseemly brawl in which Andy Cole was sent off. Souness accused Rofe of trying to influence the officials, although he later had to accept that the sending off was justified.
In December 2005, following the appointment of George Burley as manager, Rofe was dismissed along with most of the coaching staff at St. Mary's.
Since leaving Southampton, Rofe has been engaged as a match summariser on BBC Radio 5, and spent the summer of 2006 working in the Bahamas with Luther Blissett at the annual Premier League Soccer Camp.
He was working as the Football League's regional youth development officer until June 2012 before joining League One side AFC Bournemouth in 2012 as first-team coach working under management team Paul Groves and Shaun Brooks. On 3 October 2012, Groves was sacked as manager of the club, and Rofe took over as caretaker. He was unceremoniously sacked on 15 October, whilst Groves and Brooks were allowed to return to their former jobs with the youth team at Bournemouth.