Richards was recruited from Xavier College, with the 27th pick in the 2000 national draft. He made his senior debut in round 2, 2002 against Richmond and went on to play 33 games, before being traded to the Sydney Swans at the end of the 2005 season.
In round 6, 2005, he opposed Jonathan Brown, who was returning from a five-match suspension from the 2004 AFL Grand Final; Richards had eight goals kicked against him, and was dropped for the following round. It was his limited opportunities at the club which ultimately led to him being traded.
He was traded to Sydney Swans in exchange for pick 19 and 50 and made his debut for them in the first round of 2006, against his old club, Essendon. In the second quarter, coach Paul Roos assigned him onto Matthew Lloyd, who had already kicked six first-quarter goals. The change paid dividends, as Richards kept his former leader goalless in the second quarter. Despite this, his old club Essendon, defeated his new club the Sydney Swans by 27 points, in what turned out to be their only win for the first 16 rounds of the season. He became a key contributor in the Swans' defense, occasionally moving forward if required.
Richards played in the 2006 AFL Grand Final and put in a good effort, despite being knocked unconscious in the final five minutes, and is said to have lost memory of parts of the game.
In 2009, he suffered a punctured lung and broken ribs when he collided with Brendan Fevola in the final 30 seconds of the loss against Carlton at Etihad Stadium. He missed the rest of the season as a result, while Sydney missed the finals for the first time in seven years. He also was forced to travel back to Sydney by road.
The following year Richards initially looked like he would struggle to win a consistent spot in the senior team. However an injury to Craig Bolton provided a spot for him. His season was capped off with the Best Clubman Award. At the end of the year he completed a Bachelor of Commerce degree at the University of New South Wales, and combined with his football career at the Sydney Swans he now works part-time in the banking industry whilst studying for a Master of Finance degree.
After a particularly strong end to the 2010 season, he continued to play consistent football throughout the following season. He became the clubs undisputed best defender and would take on the top forwards around the AFL every week, rarely being beaten.
Richards continued on his upward climb throughout the 2012 season; he was selected in the All-Australian team and was an integral part in Sydney winning the 2012 AFL Grand Final. He established himself as one of the best defenders in the competition and finished second in the Bob Skilton Medal.
He announced his retirement on 5 August 2016, and said he would finish at the end of the season.