Brown attended Joseph Wheeler High School in Marietta, Georgia. As a senior, he helped lead his team to victory in the Georgia High School Association (GHSA) AAAAAA state championship. With 0.6 seconds remaining, Brown hit two free throws to give Wheeler a 59–58 win. As a senior, Brown averaged 28 points and 12 rebounds while leading Wheeler to a 30–3 overall record.
Brown was rated as a five-star recruit and was ranked by Scout.com, ESPN, and 247Sports.com as the fourth best recruit in his class behind Ben Simmons, Skal Labissière, and Brandon Ingram. Rivals.com ranked him third in his class.
Brown won a 2014 FIBA Americas Championship gold medal as part of the USA Basketball Men's U18 National Team. Brown was also selected to play in the 2015 McDonald's All-American Boys Game. At the conclusion of an outstanding high school career, Brown was named Gatorade Georgia Boys Player of the Year, USA Today's All-USA Georgia Player of the Year, Georgia's Mr. Basketball, and the Class 6A Player of the Year.
On May 1, 2015, Brown committed to play for the Golden Bears at the University of California, Berkeley, under coach Cuonzo Martin and alongside fellow top-recruit Ivan Rabb. He was heralded as an all-around prospect due to his athleticism. Brown took a postgraduate class during his first semester in college. The class was part of a Master’s degree program on Cultural Studies of Sport in Education. In addition to that, Brown also learned how to speak Spanish relatively well in his time at California, with him planning on learning three more languages by the age of 25.
While playing for California, Brown averaged 14.6 points, 5.4 rebounds and 2.0 assists in 27.6 minutes per game over 34 games. He had his best scoring games on November 27, 2015, against Richmond and January 27, 2016, against Utah by recording 27 points in each game. He recorded a season-high 11 rebounds twice during victories on November 23, 2015, against Sam Houston State and on January 1, 2016, against Colorado. On January 23, 2016, Brown recorded a season-high 7 assists to go with 15 points in a 74–73 victory over Arizona. Brown earned first-team All-Pac-12 honors and was named the Pac-12 Freshman of the Year.
On June 23, 2016, Brown was selected by the Boston Celtics with the third overall pick in the 2016 NBA draft. On July 27, 2016, he signed his rookie scale contract with the Celtics after averaging 16.0 points, 6.2 rebounds and 2.3 steals in six Summer League games. He made his debut for the Celtics in their season opener on October 26, 2016 against the Brooklyn Nets, scoring nine points on 3-for-4 shooting, while adding two blocked shots in 19-plus minutes. In his first career start on November 3, 2016, Brown scored 19 points in a 128–122 loss to the Cleveland Cavaliers. On January 27, 2017, he scored a career-high 20 points in a 128–98 win over the Orlando Magic. Brown helped the Celtics claim the No. 1 seed in the Eastern Conference, before helping them advance through to the Eastern Conference Finals, where they were defeated by the Cavaliers in five games. Brown had a productive rookie season in 2016–17, with his role off the bench continuing to develop as the year went on. He appeared in 78 games for the Celtics during the regular season, with 20 starts. He averaged 17.2 minutes on the floor each night and he put up 6.6 points with 2.8 rebounds and 0.8 assists. At the season's end, he was named to the NBA All-Rookie Second Team.
Brown has been described as a "scholar" and "introvert". He is primarily a vegetarian and strives to obtain a diversity of interests including learning Spanish, studying history, watching soccer, meditation, and philosophy. Many have described him as an unusual athlete, with many ambitions beyond basketball. Brown, who is African-American, assembled a primarily African-American advisory team prior to the NBA draft, though he did not hire an agent. He was criticized by some as "too smart" to play in the NBA. This criticism was taken by some as racial bias against African-Americans.
Brown's father is Quenton Marselles Brown, a professional boxer, who is the 2016 WBU World Champion, the 2015 WBU C.A.M. Heavyweight Champion as well as a member of the Hawaii State Boxing Commission Board.
Brown also has a YouTube channel, where he posts several documentary style video series showcasing his life during the season and off-season workouts. The first episode FCHWPO: Pawn to E4 was posted on January 31, 2017. The video title refers to Brown's love for the game of chess. FCHWPO, which is also Brown's Twitter and Instagram handle, is an acronym for Faith, Consistency, Hard Work Pays Off.