Melichar was drafted in the 3rd round (71st overall) by the Penguins in the 1997 NHL Entry Draft. He played two junior seasons with the Tri-City Americans of the Canadian Hockey League (CHL) from 1997 to 1999, and was nominated for the Western Hockey League (WHL)'s top defenceman award in 1998. The award would go to future Penguins teammate and frequent defense partner Michal Rozsíval.
Melichar's first pro season was spent with the Wilkes-Barre/Scranton Penguins of the American Hockey League (AHL) in their inaugural 1999–00 season. He was the only member of the team to play in all 80 games.
Melichar made his NHL debut as a 21-year-old in October 2000, and played in 18 games for the Pittsburgh Penguins. The Penguins were stacked with Czech players that season, with Melichar being one of eleven to suit up for the team. He was sent down to Wilkes-Barre/Scranton and participated in their long playoff run, which ended with a loss in the 2001 Calder Cup finals.
He became a permanent fixture on Pittsburgh's blueline in 2001–02, with a strong rookie season that ended prematurely with a shoulder injury. After being rewarded with a one-way contract, Melichar only played eight games in the 2002–03 season due to re-aggravation of the shoulder injury.
After several shoulder surgeries, Melichar came prepared for training camp in 2003, only to injure the same shoulder yet again. However, the injury was not as severe as expected, and he was able to play in all 82 games in the 2003–04 season. He also scored his first NHL goal that season (in his 137th NHL game) against John Grahame and the Tampa Bay Lightning on January 27, 2004 in a 6-2 Penguins loss.
After playing for HC Sparta Praha in the Czech Republic during the 2004–05 NHL lockout, he signed a multi-year contract with Pittsburgh on August 15, 2005. He was named an alternate captain for the 2005–06 Penguins on March 11, 2006. Melichar finished the season with 3 goals and 12 assists for a career high 15 points in 72 games.
Melichar in his sixth and final season with the Pittsburgh Penguins, was partnered on the 2nd defensive pairing with Rob Scuderi. He was second on the team in blocked shots and maintained a strong plus-minus rating throughout the season, while playing against opposing teams' top players. Without a contract offer from the Penguins, Melichar left for Europe and played with HC České Budějovice in the Czech Extraliga and Linköpings HC in the Swedish Elitserien.
On July 1, 2008, Melichar signed with the Carolina Hurricanes. Melichar was then traded by the Hurricanes, along with Wade Brookbank and a fourth round draft pick in the 2009 NHL Entry Draft, to the Tampa Bay Lightning for Jussi Jokinen on February 7, 2009. He then left his club the Tampa Bay Lightning in July 2009 and he then turned back to his hometown České Budějovice and signed for HC České Budějovice.
On August 10, 2010, Melichar signed a contract with Linköpings HC, marking his second tenure at the Swedish club.
On July 5, 2011, Melichar signed with HC Pardubice, making it his fourth time that he has played in Pardubice.
Melichar abruptly announced his retirement on August 18, 2011, only a month after signing his one-year contract. A statement released from HC Pardubice read: "After a long deliberation, taking into account his family situation, [Josef Melichar] decided this season to retire from hockey for personal reasons and asked us for early termination of the recently signed professional contract with our club."
Michal Sivek, Melichar's agent and former teammate, would not elaborate on Melichar's retirement other than saying "He simply has no desire to play hockey."2007 – Signed as free agent by Linköpings HC in Elitserien
July 2, 2008 – Signed 1 year, $1 million contract with the Carolina Hurricanes for the 2008–2009 NHL season.
February 7, 2009 – Traded, along with Wade Brookbank and a fourth round draft pick in the 2009 NHL Entry Draft to the Tampa Bay Lightning for Jussi Jokinen