Parisé was signed by the Boston Bruins at 21, after a scout saw Parisé score four goals and two assists in a playoff-clinching game, and was assigned in to the Bruins' junior league club, the Niagara Falls Flyers of the Ontario Hockey Association. On the Flyers, he played for former NHL player and future Bruins general manager Hap Emms. While he was not a promising scorer in juniors, he learned to play a diligent two-way game and became noted as a skilled penalty killer. He turned pro the following season and, in the days of the Original Six when big league jobs were few, spent most of the next five seasons in the Bruins' farm system. He started to find his scoring touch in 1964 with the Minneapolis Bruins of the Central Professional Hockey League, scoring 63 points in 72 games, and was named a Second Team league All-Star with the Bruins' Oklahoma City Blazers affiliate in 1966.
He made his NHL debut the same season with the Boston Bruins, playing limited action in three games, followed by eighteen games the next season.
The following season saw expansion, and Parisé was drafted by the Oakland Seals on June 6, 1967. On October 12, he was traded to the Toronto Maple Leafs for Bryan Hextall Jr. for Gerry Ehman and assigned to the Rochester Americans of the American Hockey League. He would play three for the Amerks (and one for the Leafs), before being dealt again on December 23, this time to the Minnesota North Stars along with Milan Marcetta for Murray Hall, Ted Taylor, Len Lunde, Don Johns, Duke Harris, and the loan of Carl Wetzel.
Playing on a line with center Jude Drouin and high scoring right winger Bill Goldsworthy, Parisé finally became a star, playing six seasons and parts of two others in Minnesota. He was named twice to play in the NHL All-Star Game, and had his best professional season in 1972–73, when he scored 27 goals and 75 points.
The degree to which Parisé was held in respect in the hockey world resulted in being named to play for Team Canada in the Summit Series in 1972. He was a surprise pick to the team and didn't expect to see much ice time, but ended up playing in six of the eight games. He played on a line with superstar Phil Esposito and had two goals and two assists.
Parisé was best known in the series for nearly attacking unpopular referee Josef Kompalla with his stick in the eighth game after Kompalla had handed out a series of questionable penalties against the Canadians. Parisé held back at the last minute from striking Kompalla and was ejected from the game in consequence. As a silver lining, the officiating of the game improved considerably after Parisé's threat which allowed his teammates a chance to get back in the game.
At age 34, the North Stars felt that Parisé was aging, and dealt him to the New York Islanders midway through the 1975 season for Doug Rombough and Ernie Hicke. He then went on to be one of the key players to not only lead the Islanders to their first playoff berth that season but all the way to the Stanley Cup semifinals, where the Isles lost in a hard-fought seven game series to the eventual Cup champion Philadelphia Flyers. Parisé scored 16 points in 17 playoff games that year, second to former North Stars teammate Drouin, who had been acquired in a separate deal with Minnesota that season. Parisé played two and a half more seasons on Long Island, adding over 20 goals each of his full seasons and providing excellent defensive play.
Halfway through the 1978 season, Parisé was traded to the Cleveland Barons along with Jean Potvin for Wayne Merrick, Darcy Regier, and Cleveland's fourth-round choice in the upcoming 1978 draft. After the demise of the Barons and their June 5 merger draft with the Minnesota franchise, Parisé became a North Star again and he would serve as the team's captain while playing his final season before retirement.
In total, Parisé played 890 games in the NHL (leading the league in games played in three seasons), scoring 238 goals and 356 assists for 594 points, adding 706 penalty minutes. He also had 27 goals and 31 assists in 86 playoff games.
After his retirement, he served as a coach in the North Stars' organization, as assistant coach between 1980 and 1988, except for the 1984 season, when he was the head coach for Minnesota's minor league affiliate, the Salt Lake Golden Eagles of the Central Hockey League; he recorded a 35–35–2 mark as coach. Thereafter Parisé retired to Minnesota where he coached and was hockey director at Shattuck-Saint Mary's, where his son Zach Parise and many other players such as Sidney Crosby, Jack Johnson and Jonathan Toews played.
On April 23, 2008, Parisé was named the head coach and general manager of the Des Moines Buccaneers of the United States Hockey League. He served one season as the team's coach.
It was reported in November 2014 that Parisé was in the advanced stages of lung cancer. Parisé died on January 7, 2015, of lung cancer, aged 73. In addition to son Zach, J.P. Parisé was survived by his wife of 42 years, Donna, two other sons, one daughter, and six grandchildren.Named to play in the NHL All-Star Game in 1970 and 1973.
On April 11, 1975, Parisé scored a goal 11 seconds into overtime to give the Islanders a 4–3 win over the New York Rangers, clinching the best-of-3 playoff series for the Isles. It was the fastest OT goal in NHL history up to that time.
At the time of his retirement, Parisé was the North Stars' all time leader in assists, second in points, third in goals and fifth in games played.
Parisé's youngest son Zach is a current NHL player with the Minnesota Wild (playing in the same market formerly occupied by the North Stars), and his eldest son Jordan played professional hockey in Europe.