Born in Shibata, Niigata, Togashi moved to the United States in 2009 where he attended Montrose Christian School in Rockville, Maryland. During his three-year career at Montrose, he played on the Mustangs' nationally ranked teams including the 2011 squad that captured the ESPN RISE National High School Invitational (NHSI) with a double-overtime victory against Oak Hill Academy. He graduated in 2012, and after not receiving any NCAA Division I scholarship offers, he returned to Japan.
In January 2013, Togashi signed with the Akita Northern Happinets of the bj league. In his debut game on 2 February 2013, he recorded 15 points, 11 assists, 3 rebounds and 3 steals in a win over the Toyama Grouses. He went on to play 30 games for the Happinets in the second half of the 2012–13 season, averaging 14.5 points, 6.2 assists, 2.9 rebounds and 1.2 steals in 37.2 minutes per game.
Togashi re-joined the Akita Northern Happinets for the 2013–14 season where he managed his first full professional season. In 58 games, he averaged 16.3 points, 7.6 assists, 2.9 rebounds and 1.1 steals in 36.8 minutes per game. He subsequently became the youngest player to be named to the bj league's Best Five Team, and was named the MVP of the league's All-Star game, held in January 2014.
In July 2014, Togashi joined the Dallas Mavericks for the 2014 NBA Summer League where he was the shortest player in the competition and subsequently became a fan favorite. In four summer league games, he averaged 4.0 points and 1.5 rebounds in 9.3 minutes per game. On 15 October 2014, he signed with the Mavericks, only to be waived by the team six days later.
On 1 November 2014, Togashi was selected in the second round of the 2014 NBA Development League Draft by the Santa Cruz Warriors. He was later traded to the Texas Legends on draft night, the Mavericks' D-League affiliate team. He officially joined the Legends on 20 November 2014, and made his debut the following day, playing four minutes and recording one steal in a loss to the Santa Cruz Warriors. On 20 February 2015, Togashi suffered a left ankle injury which later ruled him out for the rest of the season. A month later, he was waived by the Legends. In 25 games for the Legends, he averaged two points and one assist in 8.3 minutes per game.
On 12 August 2015, Togashi signed a try-out contract with Italian Serie A side Dinamo Banco di Sardegna Sassari. He later parted ways with Sassari on 20 September 2015 before appearing in a game for them.
On 9 October 2015, Togashi signed with the Chiba Jets of the Japanese National Basketball League.
In 2014, Togashi was a member of the Japanese national team that competed in the William Jones Cup and the Asian Games, winning bronze in the latter tournament.