Downing was born in Trenton, New Jersey. He participated in the Police Athletic League. Downing attended Trenton Central High School, Muhlenberg College in Allentown, Pennsylvania, and Rider College in Lawrenceville, New Jersey. He also played baseball as a semi-professional.
Downing signed with the New York Yankees as an amateur free agent in 1961, and was promoted to the major league roster by July of that season. In 1963, his first full major league season, Downing had a 13–5 win-loss record with a 2.56 earned run average (ERA) for a Yankee team that went 104–57, but were swept by the Los Angeles Dodgers in the 1963 World Series. In 1964, he went 13–8 with a 3.47 ERA, and led the league with 217 strikeouts.
Downing was 9–5 with a 2.66 ERA when he made his only All-Star team in 1967. He pitched two innings, giving up no earned runs while striking out two. On August 11, he struck out three batters (Tony Horton, Don Demeter and Duke Sims) on nine pitches in the second inning of a 5–3 win over the Cleveland Indians. Downing became the sixth American League pitcher and the 13th pitcher in Major League history to accomplish this feat.
Injuries limited Downing to only twelve starts in 1968. In 1969, Yankees manager Ralph Houk began using Downing out of the bullpen more, as he made fifteen starts and fifteen relief appearances. He was traded to the Oakland Athletics prior to the 1970 along with catcher Frank Fernández for Danny Cater and Ossie Chavarria.
Oakland traded Downing and Tito Francona to the Milwaukee Brewers on June 11, 1970, for Steve Hovley. Despite a respectable 3.34 ERA, Downing's record was 2–10 for a Brewers team that narrowly escaped losing 100 games (97). For the season, Downing went 5–13 with a 3.52 ERA and 79 strikeouts in 27 games and 22 starts between his two teams.
Prior to the start of the 1971 season, the Brewers traded Downing the Los Angeles Dodgers for Andy Kosco. In his first season in the National League (NL), Downing won twenty games, and pitched a league-leading five shutouts. He earned NL Comeback Player of the Year honors as well as finishing third in NL Cy Young Award balloting behind Ferguson Jenkins and Tom Seaver.
On April 8, 1974, Downing allowed a home run to Hank Aaron that was the 715th of his career, breaking the all-time record set by Babe Ruth. Downing made his third, and final post-season appearance that season. His Dodgers lost four games to one to the Oakland A's. Downing played two more full seasons with the Dodgers, and was released during the 1977 season with a 0–1 record and 6.75 ERA.
Downing served as a color analyst on Dodgers cable-TV broadcasts from 1980–87 and on Dodgers radio in 2005. He also broadcast for CBS Radio in the 1990s, and the Atlanta Braves in 2000. As of 2006, he remains on the Dodgers Speaker's Bureau.