A product of the Tottenham Hotspur youth academy, Prosser joined Port Vale in 2005. He spent five years at the club, also spending time on loan at Leigh RMI, Salisbury City, and Kidderminster Harriers. He moved on to Southend United in 2010, and was initially loaned out to Rushden & Diamonds. He played for Southend in the 2013 Football League Trophy Final and was an unused substitute in the 2015 League Two play-off final victory. He was loaned out to Northampton Town in January 2016, and helped the club to win promotion as League Two champions in 2015–16. After being released by Southend, he joined Essex rivals Colchester United in June 2016.
Born in Enfield, London, Prosser was signed by Port Vale from the Tottenham Hotspur youth system. He played in the Vale youth setup as a trainee before signing a professional contract on 1 July 2006. He was offered a new six-month contract by Vale in May 2007. He was loaned to Leigh RMI in 2008 to gain first team experience, where he made seven league appearances. After returning from this loan spell, he made his debut for Port Vale in a 3–2 victory over Brighton & Hove Albion on 5 April 2008. He finished the 2007–08 season with five appearances for the club, after which he was offered a new contract.
Following an injury to Gareth Owen in March 2009, Prosser was given an opportunity to affirm his place in the team, and hoped to earn a new contract at the end of the 2008–09 season. He succeeded in this aim, signing a 12-month deal in June 2009. However, two months later he was transfer listed by new manager Micky Adams in August 2009 along with five other young players, having failed to impress in pre-season. He swiftly joined Conference National side Salisbury City on a one-month loan deal along with teammate Danny Glover. The pair made their debuts as substitutes in a 1–0 defeat to Mansfield Town at the Raymond McEnhill Stadium on 15 August. After playing six games for Salisbury, Prosser's loan was extended for a further month on 8 September. He was recalled by Adams in October 2009, though he had to settle for a place on the bench, behind the first choice centre-halves. Another injury to Gareth Owen allowed Prosser to return to the first eleven for a league clash with Rotherham United on 14 November. In an eventful first half, Prosser scored his first professional goal with a close ranger header, before being sent off for two bookable offences. In March 2010, he was informed that he would not be offered a new contract by Port Vale at the end of the season. He immediately joined Conference side Kidderminster Harriers on loan until the end of the season, where he made nine appearances.
In search of a new club, Prosser joined Barnet and Lincoln City on trial ahead of the 2010–11 season. With a transfer embargo in situ, he signed a pre-contract with Southend United following a trial in August 2010. In January 2011, after playing only 13 matches for Southend, Prosser was loaned to Rushden & Diamonds for three months. He was recalled from his loan on 18 February, after playing six games for the Conference club and scoring only his second career goal. Back at Southend, Prosser earned a recall against his former club Port Vale on 9 April, where his last minute goal earned Southend a point. He signed a new one-year deal with the club the following month.
Prosser made 27 appearances for Southend in all competitions during 2011–12, as the club narrowly missed out on automatic promotion and failing in the play-offs. With his contract expiring, Southend activated a 12-month extension clause in Prosser's contract in June 2012. The following season, Prosser captained Southend in Chris Barker's absence during a 1–1 draw with Accrington Stanley on 9 February 2013. On 7 April, he featured at Wembley Stadium for the club in the 2–0 Football League Trophy final defeat to Crewe Alexandra. In June 2013, Prosser rejected an initial offer of a new contract from Southend. However, a week later, Prosser agreed a new two-year deal with the club.
Prosser made 31 and 33 appearances respectively in the two following seasons as Southend twice reached the play-offs, faltering at the semi-final stage in 2013–14 but reaching the final in 2014–15. During the campaign, he was knocked unconscious by a free kick from Cambridge United's Ryan Donaldson on 21 March 2015. He was an unused substitute in the play-off final as Southend beat Wycombe Wanderers to earn promotion to League One.
Northampton Town were keen to sign Prosser ahead of the 2015–16 season, but had a transfer bid rejected in June 2015. However, he found himself out of favour at Southend, and on 28 January 2016, he joined Northampton on loan until the end of the season. He helped Northampton to the League Two title as he made eight league appearances. He was not offered a new contract to remain with his parent club, and he was released by Southend in May 2016.
Prosser signed with League Two side Colchester United on a two-year contract in June 2016. He was named as captain by manager John McGreal ahead of the 2016–17 season. He made his competitive debut for the club on 6 August in Colchester's 1–1 draw with Hartlepool United at Victoria Park on the opening day of the season. After making 17 appearances for the Colchester United first team, Prosser was ruled out for the remainder of the 2016–17 season in November 2016 following surgery on his knee.
In July 2017, Prosser underwent a second operation on his knee that would leave him out of action for a further two months.
Prosser has a younger sister, Abbie, who is also a footballer. She has represented England at under-20 level and had been capped 18 times at under-19 level. She played at club level for Arsenal, Barnet, and Doncaster Rovers Belles.
In July 2009, a "momentary lapse of concentration" led to Prosser reaching 108 mph on the M6 Toll road. As a result, was banned from driving for seven days and fined £225 plus £50 costs.As of match played 5 November 2016.
2013 Football League Trophy Final runner-up
2015 Football League Two play-off Final winner
2015–16 Football League Two champion