Brown worked his way through the youth team of Queens Park Rangers, earning a professional contract in July 2008 having been part of the Football League Youth Alliance winning side the previous season.
He spent the first half of the 2009–10 season on loan at administration-stricken Salisbury City of the Conference National, along with teammate Joe Oastler. He made fourteen appearances for the club, including twelve starts, before returning to Loftus Road before the end of the calendar year. On 24 April 2010 he made his debut in the Football League, replacing Tamás Priskin as a substitute in a 1–0 win over Barnsley at Oakwell.
At the end of the season Neil Warnock offered him a new one-year deal, which he duly signed. "Lee Brown has emerged as an exciting prospect from the QPR youth system in recent years and was rewarded with a one-year deal at QPR in 2009. He, along with several youngsters were given the opportunity to train with the first team in the latter part of the 2008–09 season, after impressing in the reserves. The left back played an important role in QPR's pre-season of 2009–10 and was subsequently given his first squad number in light of the lack of full backs at the club. Brown, along with fellow defender Joe Oastler, was loaned out to Conference Premier side Salisbury City. His stay there was curtailed due to Salisbury being unable to renew their loan deals"
In September 2010, Brown joined Hayes & Yeading United on loan, initially for one month before being extended for a second. Brown returned to Church Lane on transfer deadline day in a loan deal until the end of the 2010–11 campaign, snubbing a similar move to League One side Leyton Orient.
Brown was released by QPR following the 2010–11 season and went on to sign a 2-year contract with recently relegated League Two outfit Bristol Rovers. He made his debut in their opening game of the season, live on Sky Sports, against AFC Wimbledon. Rovers won the game 3–2. In his first season at Bristol Rovers he appeared in 42 league games and scored seven goals for Rovers as the club turned around a difficult first half to the season, which resulted in the sacking of manager Paul Buckle who was replaced by Mark McGhee, to finish in a comfortable, albeit disappointing, 13th place.
On 18 July 2012, Brown signed an extension to his Bristol Rovers contract to keep him at the Memorial Stadium until the summer of 2014. He was again a regular for Rovers as he made 39 league appearances, scoring three times. For Rovers, the season was again a struggle for the first half, spending two different spells in the relegation zone leading to the dismissal of Mark McGhee. He was replaced by the returning John Ward who successfully guided Rovers to 14th in League Two.
Brown's third season at Bristol Rovers started with the club among the pre-season favourites for promotion. The season proved to be a disaster for Rovers as the club was relegated out of the Football League for the first time since joining it in 1920. Brown made 41 appearances, scoring twice.
Despite attracting interest from Football League sides, Brown signed a new contract with Rovers. Brown stated "I thought I had to give something back to [the fans]" and that manager Darrell Clarke, who had replaced John Ward toward the end of the previous season and had played alongside Brown at Salisbury City in 2009, "has got to take a lot of credit for me staying". He made his international debut, representing England C in October's 2–0 defeat to Turkey B in Istanbul. Brown achieved promotion to the Football league via a play-off victory in the 2015 Conference Premier play-off Final.
Arguably his best season ever as a footballer, Lee Brown played every minute of Rovers' games. He scored 6 goals over the course of the season after opening his tally in a home win against Barnet. On the 14th April, Brown signed a new and improved deal for the club. His last goal of the season proved to be the most important as his 92nd-minute winner on the last game of the season against Dagenham & Redbridge made Rovers a Football League One side once again.
Brown scored his first goal of the season in a FA Cup first round tie against Crawley Town striking a superb shot from 35 yards into the bottom left corner.
Brown started on the opening day in a 1-0 defeat to Charlton Athletic. He also played the following Tuesday in a 4-1 Carabao Cup victory over Cambridge United in which he got two assists. A 40 yard pass to Billy Bodin for the first goal and a delightful cross to Ellis Harrison for the third goal.As of match played 16 September 2017