Colaiacovo was drafted 17th overall by the Toronto Maple Leafs in the 2001 NHL Entry Draft. He scored his first career NHL goal on November 8, 2005, against the Washington Capitals.
On January 23, 2006, Colaiacovo missed a hipcheck on Václav Varaďa in a game against the Ottawa Senators, smashing head-first into the boards. He was taken off the ice on a stretcher. The injury was a major concussion, and he missed the end of the 2005–06 season. He was then sent to the Toronto Marlies, the Maple Leafs' American Hockey League (AHL), on a conditioning assignment.
Colaiacovo recovered fully from his concussion, but later injured his hand in his first game back into the Marlies lineup. On December 4, the Maple Leafs recalled Colaiacovo from the Marlies and on December 16, he scored his first goal of the season against the New York Rangers. Colaiacovo underwent arthroscopic knee surgery on April 29, 2007.
On November 24, 2008, Colaiacovo was traded by the Maple Leafs, along with Alexander Steen, to the St. Louis Blues in exchange for forward Lee Stempniak. Colaiacovo finished the 2008–09 season with 30 points between Toronto and St. Louis. In the 2009–10 season, he registered a career-high 32 points, finishing second among Blues defencemen. After the 2010–11 season, he joined Team Canada at the IIHF World Championship in Slovakia.
On September 12, 2012, Colaiacovo signed a two-year deal with the Detroit Red Wings. In his second game of the season, Colaiacovo suffered a shoulder injury that kept him out for 33 games. He made his home debut for the Red Wings on April 1, 2013, against the Colorado Avalanche. When asked about his feelings over his home debut, Colaiacovo replied, “I'm really looking forward to it. When I first signed here I said the Joe is one of my favorite rinks to play in. I haven't had that chance to do it in a Wings uniform. That's something that's going to keep the adrenaline going for me tonight."
With the lockout-shortened 2012–13 season, again largely affected by injury, Colaiacovo was bought out on a compliance from the final year of his contract with the Red Wings on July 3, 2013.
On November 13, 2013, Colaiacovo agreed to a one-year, $550,000 deal to return to the St. Louis Blues as a free agent. The Blues added Colaiacovo after learning that defenceman Jordan Leopold would be out eight weeks due to hand surgery.
On October 30, 2014, Colaiacovo signed a deal with the Philadelphia Flyers. He was signed to a one-year, two-way contract for $625,000.
On July 3, 2015, Colaiacovo signed a one-year deal as a free agent with the Buffalo Sabres. He made 36 appearances for the Sabres in the 2015–16 season from the blueline, contributing with one goal and four assists.
On November 26, 2016, he belatedly signed as a free agent with Adler Mannheim of the Deutsche Eishockey Liga in Germany for the remainder of the 2016–17 season, embarking on his first overseas gig.
Colaiacovo's twin brother Paulo is a former professional hockey player, and his cousin, Matt Finn (born February 24, 1994), was selected by the Toronto Maple Leafs in the second round, 35th overall, of the 2012 NHL Entry Draft.