Anderson was drafted in the first round, 11th overall by the Toronto Maple Leafs in the 1977 NHL Entry Draft. He played 814 career NHL games, scoring 282 goals and 349 assists for 631 points from 1977–78 until 1988–89. His best statistical season was the 1982–83 season, when he set career highs with 49 assists and 80 points. Anderson scored the winning goal against the New York Rangers on April 4, 1987, to give the Hartford Whalers their only division championship. Anderson was the captain of his junior team, the Toronto Marlboros.
In 1996-97, Anderson coached the Quad City Mallards to their first of six consecutive 50-win seasons and their first Colonial Hockey League championship in just the franchise's second season. John Anderson is the Chicago Wolves franchise's all-time coaching leader in wins with 371 and holds the club mark for postseason victories as well with 80. John led the Wolves in winning the Turner Cup and Calder Cup four times in his eleven seasons at the team's helm. His team was crowned league champions in 1997–98, 1999–00, 2001–02 and 2007–08.
Anderson also helped establish "John Anderson's", a diner best known for its "Banquet Burger", as well as its $4 breakfast special. The original restaurant is located south of Yonge & Finch streets in Toronto, Ontario. There is another location at the corner of Dundas and Erindale Station Road in Mississauga, Ontario. The key items offered at the Mississauga location are the "Big Puck Burger" and "John's New Specialty", which is souvlaki on a sesame-seed bun. The original restaurant at Yonge & Finch has been moved to the southeast corner of Yonge and Hendon.
Anderson coached the American gold medal winning team in the 2007 Jewish World Cup hockey tournament in Israel.
On June 20, 2008, Anderson was named as the fourth head coach of the Atlanta Thrashers.
On October 10, 2008, Anderson won his first game as an NHL coach 7–4 against his good friend Bruce Boudreau's Washington Capitals.
On April 14, 2010, Anderson was released as head coach of the Atlanta Thrashers after 2 seasons with the organization.
On July 12, 2011, Anderson became an assistant coach for the Phoenix Coyotes.
On July 10, 2013, Anderson was rehired as the head coach of the Chicago Wolves.
On June 8, 2016, Anderson was hired as an assistant coach for the Minnesota Wild.
Anderson played for Canada at the 1977 IIHF World U-20 Championship.