He began his career at Leyton Orient, before winning a £100,000 move to Oxford United in July 1996. He continued to impress, and was bought by Birmingham City for £700,000 in February 1998. He played for the "Blues" in the Premier League and the 2001 Football League Cup Final, before switching to West Bromwich Albion for a £750,000 fee in June 2004. After a year with the "Baggies", he was transferred to Championship side Cardiff City for just under £1 million.
Appointed as club captain, he was an unused substitute in the 2008 FA Cup Final, where they were defeated by Portsmouth. He joined Sheffield Wednesday in July 2009, and was again appointed as captain. The "Owls" suffered relegation into League One, and he moved on to Millwall in January 2011. Loaned out to Yeovil Town, he switched to Plymouth Argyle for a 12-month spell in January 2012. He left Plymouth for Port Vale in January 2013, and helped the club to secure promotion out of League Two in 2012–13. He left England in August 2013 to play for Finnish club IFK Mariehamn, before heading into English non-League football with Chesham United, Welling United, Staines Town, AFC Rushden & Diamonds, Hayes & Yeading United, Bishop's Stortford, AFC Rushden & Diamonds, and Enfield Town.
Purse was born in Stepney, London and attended Cardinal Pole RC School in Hackney. As a youngster Purse played in the same Sunday league side as Lee Bowyer. He signed his first professional contract with Leyton Orient when he turned seventeen, and made his début just three days later. He spent close to two seasons at Brisbane Road, and scored five goals in 66 league and cup appearances for the club. Orient were relegated out of the Second Division in last place in 1994–95, and finished 21st in the Third Division in 1995–96. He also spent time in Finland on loan at Kakkonen (Second Division) club BK-IFK.
In July 1996, Purse was signed by Oxford United for a £100,000 fee. He made 31 appearances in the 1996–97 season, as Denis Smith's "Yellows" posted a 17th-place finish in the First Division. Purse impressed at the Manor Ground, and scored six goals in 35 games in 1997–98.
Purse joined Birmingham City for £600,000 plus Kevin Francis (valued at £100,000) in February 1998. He started just two First Division games at the end of the 1997–98 season. He made 11 league starts and nine substitute appearances in the 1998–99 campaign, and also appeared at St Andrew's as a substitute in the play-off defeat to Watford after David Holdsworth was sent off – he converted a penalty in the shoot-out, which ended in a 7–6 victory for Watford.
Purse established himself in the "Blues" first team in the 1999–2000 season, making 41 starts in league and cup competitions. He played at the Millennium Stadium in the 2001 Football League Cup Final, and scored a 90th-minute penalty to take the game into extra-time. He also converted his penalty in the shoot-out, which was won by Liverpool 5–4. With four goals in 49 appearances in the 2000–01 season, Purse won the vote to be named as the club's player of the season. This represented a significant turnaround for a young defender who was barracked by City supporters at the start of the campaign.
He missed the start of the 2001–02 season after a training ground shot from manager Trevor Francis broke his wrist. Francis was sacked just after Purse returned to full fitness, though the defender managed to keep his first team place under new boss Steve Bruce, and started 40 matches in the 2001–02 season.
In August 2002, Purse "ended months of speculation" by signing a three-year deal (with an option for a fourth year). Having been courted by Arsenal, Charlton Athletic and Everton, he had previously been stripped of the club captaincy after twice rejecting Birmingham's offer of a £10,000 per week five-year contract. The protracted negotiations were put down to his ambitions for top-flight football, which were secured with victory in the 2002 play-off Final (Purse did not feature in the matchday squad). However, he missed a large chunk of the 2002–03 season after his ankle became infected following surgery.
Purse had a quiet 2003–04 campaign, making only nine Premier League appearances. He did, however, hit the headlines after elbowing Chelsea striker Adrian Mutu; the Romanian reportedly launched a "volley of expletives" at Purse in a post-match dressing room incident.
Purse moved to Gary Megson's West Bromwich Albion for £750,000 in June 2004. On signing with the club he said "I've probably got the best years of my career ahead of me." He spent a single season with Albion in the Premier League, making 24 appearances in all competitions, and was frozen out of the first team from February onwards. He therefore played no part in the "Survival Sunday" victory over Portsmouth that ensured the "Baggies" retained their top-flight status. He had made "significant errors" on the pitch, and was deemed surplus to requirement at The Hawthorns by new boss Bryan Robson.
In July 2005, Purse moved to Cardiff City in a deal worth just under £1 million. Five players (Alan Lee, Rhys Weston, Tony Warner, Chris Barker and Neil Alexander) accepted "substantial reductions in their wages" to allow the club to meet Purse's wage demands. He was appointed team captain, and forged a reputation for hard work and consistency as the foundation of the Cardiff defence, forming an effective partnership with Glenn Loovens. He scored six goals in 43 games in 2005–06 and scored four goals in 33 games in 2006–07. Purse signed a one-year extension to his contract at Ninian Park in July 2007.
"I love being here. I enjoy turning up for training every day, playing for a great manager and the team that I'm playing for and team-mates are fantastic. We've got a new stadium coming and things are looking up."
In August 2007, he received an injury to his knee cartilage and it was expected that he would be unavailable for up to four weeks, but he contracted an infection during an operation on the injured knee that delayed his return for a much longer period. He made his return to first team football in October, and went on to score a headed goal in the 2–1 League Cup defeat by Liverpool at Anfield on 31 October. The 2007–08 season turned into a disappointment for Purse as he was forced out of the side by "Bluebirds" fan favourite Roger Johnson. He also scored several own goals, including a spectacular volleyed pass back that proved too powerful for Cardiff goalkeeper Peter Enckelman. He was also labelled a "muppet" by Burnley forward Andrew Cole, after Purse was sent off for what was deemed a dangerous tackle against Cole, although Cardiff manager Dave Jones defended the player and the ban was later overturned. Purse did not feature in City's FA Cup run to the final at Wembley, and was an unused substitute as Cardiff were beaten 1–0 by Portsmouth.
Purse began the 2008–09 season on the bench until Glenn Loovens was sold to Celtic, and he returned to the starting line-up alongside Roger Johnson in September. He spent three months in the first team before losing his place in the side again, this time to Hungary international Gábor Gyepes who was signed as a replacement for Loovens, after a 2–1 defeat to Plymouth Argyle on 22 November after which Cardiff manager Dave Jones slammed the defensive performance of the team. Purse responded by criticising his manager and stating that he did not deserve to be dropped from the side. Purse was told that he could leave the club if a replacement could be signed during the January transfer window and was allowed to open talks with fellow Championship side Norwich City, but Cardiff could not find a suitable replacement in time and the move was cancelled. Able to secure a play-off place on the last day of the 2008–09 season, Cardiff instead suffered defeat and missed out on the play-offs on goal difference. At the end of the season Purse was not offered an extension to his contract and left the club.
Purse signed a two-year contract with Championship club Sheffield Wednesday in May 2009. He was immediately named as team captain. He made 41 appearances in the 2009–10 season, only missing a handful of games due to a hamstring injury. The "Owls" struggled, and manager Brian Laws was sacked and replaced by Alan Irvine. On the final day of the season, a win over Crystal Palace at Hillsborough would ensure survival at Palace's expense. Purse scored an 87th-minute goal to level the game at 2–2, however no more goals followed and Wednesday were relegated into League One.
He made 30 appearances in the first half of the 2010–11 season, before he left the cash-strapped club by mutual consent in January 2011.
The day after his release from Sheffield Wednesday, 21 January 2011, Purse signed with Millwall – the club he supported as a boy – on a contract until the end of the 2010–11 season. Purse made his début for Millwall two days later in a 4–2 defeat away at Leicester City. He scored his first goal for the club on 23 April, in a 4–0 win over Preston North End at The Den, and dedicated the goal to his father. He signed a one-year contract with Kenny Jackett's "Lions" in April 2011.
Having found himself out of the first team at the start of the 2011–12 campaign, Purse joined Yeovil Town on a one-month loan in October in the hope of proving himself worthy of a starting place at Millwall. He made five appearances during that time and "Glovers" manager Terry Skiverton stated that he would have liked to extend his stay at Huish Park.
In November 2011, Purse moved to Plymouth Argyle on loan with a permanent 18-month contract to be signed when the transfer window opened the following January. He ended the 2011–12 campaign with 24 appearances for the "Pilgrims", as the team's "solid" defence helped to keep the club in the Football League. He later said that helping Plymouth to stay in the league was "one of the best things I have done in my career".
At the start of the 2012–13 season, Purse was named as the club's new captain by manager Carl Fletcher. The 35-year-old said he would be "honoured" to captain the club. In December, he was transfer listed "with some reluctance" in the hope of releasing funds for recruitment of new players in the January 2013 transfer window. In January 2013, he was reported to have begun talks with League Two club Port Vale. He left Home Park by mutual consent on 4 January, having made 46 appearances for Argyle in all competitions.
Purse signed for Port Vale on 5 January 2013. His signing was announced on the same day as the "Valiants" League Two clash with former club Plymouth Argyle, and only a few days after centre-back John McCombe was ruled out of action for several weeks with a knee injury. He was named 'man-of-the-match' on his debut on 12 January, as a 2–1 victory over Gillingham sent Vale to the top of the table. Vale secured promotion with a third-place finish at the end of the 2012–13 season, with Purse providing experience at the back in his 17 games. He was not offered a new contract at the end of the season and was released as a free agent, with manager Micky Adams putting this decision down to the defender's age rather than as a reflection on his performances in the 17 games.
Purse joined Northampton Town on trial in July 2013, with manager Aidy Boothroyd worried over the fitness of centre-half Kelvin Langmead. The following month he signed with Finnish side IFK Mariehamn until the end of the 2013 Veikkausliiga season. He joined Southern League Premier Division club Chesham United in January 2014, and helped the "Generals" to reach the play-off final at the end of the 2013–14 season, where they were denied promotion following a 3–1 defeat to St Albans City.
In January 2015, Purse signed for Conference National club Welling United. Two months later he moved on to Staines Town of the Conference South, scoring on his debut in a 1–1 draw at Farnborough. Staines dropped out of the division after a last place finish in the 2014–15 campaign. Purse signed with AFC Rushden & Diamonds in December 2015. He moved on to Hayes & Yeading United one month later. He played 18 matches for Hayes as they were relegated at the end of the 2015–16 season. He rejoined Chesham United in July 2016. He signed with National League South club Bishop's Stortford in March 2017. Only one month later, he left the club and joined AFC Rushden & Diamonds of the Northern Premier League Division One South. Rushden finished fifth at the end of the 2016–17 season, before losing to Witton Albion in the play-offs; Purse did not feature in the game. In May 2017, he joined Isthmian League Premier Division club Enfield Town as both a player and as assistant manager to Andy Leese.
Purse won two caps for England at under-21 level in 1998.
Speaking in June 2004, Gary Megson said that Purse is "comfortable on the ball, good in the air, strong and wants to get on and win". In December 2002, Steve Bruce said that "he's big, he's brave, he's honest and he's quick. He needs to learn to take the rashness out of his game and then he'll have a chance." Describing himself in January 2013, Purse said he was a "no-nonsense defender... [who] can play a bit of football if I need to, but I like to organise and motivate people".As of match played on 22 March 2017
Birmingham City player of the season: 2000–01
with Birmingham City
Football League Cup runner-up: 2001
with Cardiff City
FA Cup runner-up: 2008
with Port Vale
Football League Two third-place promotion: 2012–13