Born in Los Angeles, Goossen was the fourth member of a family of eight brothers and two sisters. He was a standout football and basketball player at Notre Dame High School in Sherman Oaks, California, where he graduated in 1964. Goossen is among distinguished alumni of Notre Dame such as Admiral Michael Mullen, Kirsten Dunst, Nick Folk, and Dayne Crist.
Following his graduation, the Los Angeles Dodgers signed Goossen for a six-figure bonus. He spent 1964 in the minor leagues playing first base with the Dodgers rookie-level Pioneer League team, the Pocatello Chiefs and then their single-A Florida State League team, the St. Petersburg Saints. After accepting a spring training invitation from the Dodgers, in which he shared a locker with future Hall of Famers Sandy Koufax and Don Drysdale who were on their way to winning the 1965 World Series, on 9 April the woeful New York Mets selected the 19-year-old Goossen through the first-year waiver process. Needing talented players, the Mets promoted the former bonus baby directly to the majors. Goossen batted .290 in 11 games as part of a catching group that included Chris Cannizzaro, Jesse Gonder, John Stephenson and Yogi Berra before being assigned for the rest of the season to single-A Auburn in the New York–Penn League.
On May 31, 1968, while playing for the Mets, Goossen broke up a possible perfect game by St. Louis Cardinals' pitcher Larry Jaster, hitting a single with two outs in the bottom of the eighth inning. In his time with the Mets, Goossen bounced between the majors (99 games in four years) and the minors (with all but 40 minor league games at AAA Jacksonville) and on 5 February 1969, New York traded him along with cash to the Seattle Pilots for a player to be named later (on 14 July the Pilots sent outfielder/first baseman Jim Gosger to the Mets to close the deal). Although Goossen again missed out on the possibility of getting a World Series ring, this time with the Miracle Mets, he got the only extended playing time in his career when he was called up by Seattle on 25 July, platooning as the right-handed bat opposite lefty Don Mincher at first base. Goossen posted career high numbers in average (.309), home runs (10), runs batted in (24), at bats (139), and games played (52), while catching and playing first base and left field. Although in Seattle for only two months, Goossen was there long enough to become one of the lasting characters in Jim Bouton's iconic diary, Ball Four.
After starting the 1970 season as the now Milwaukee Brewers' first baseman, Goossen's production tailed off badly, and he was sent to AAA Portland after hitting only .255 with one home run over the first 21 games. On 14 July, the Washington Senators purchased Goossen from the Brewers and he spent the rest of the season in Washington playing for Hall of Famer Ted Williams, but he hit an empty .222 with no homers and one RBI and only three extra base hits in what would be his final taste of the major leagues. On 3 November 1970, Goossen was sent to the Philadelphia Phillies with left fielder Gene Martin and relief pitcher Jeff Terpko for a player to be named later and Curt Flood, whose lawsuit for free agency was pending against Major League Baseball (on 10 April, the Phillies sent Terpko back to the Senators to complete the trade). Goosen spent the 1971 season playing for the AAA teams of three organizations, the Phillies, Chicago Cubs, and California Angels, before calling it a career following the end of the season at the age of 25.
After his baseball retirement, Goossen helped his brother, Dan Goossen who owned Ten Goose Professional Boxing along with his brothers, as a boxing trainer. Notable boxers he worked with included Rick Lindland, an amateur boxer-turned-actor and 1980s middleweight champion Michael Nunn.
While at the gym in 1988, his brother Joe asked him to meet with actor Gene Hackman, who was doing research for the film Split Decisions. Soon afterward the two became friends and the actor hired Goossen to work as his stand-in. Hackman had written into his contracts that Goossen would serve as his stand-in for every film he did. Goossen appeared in 15 of Hackman's movies between 1989 and 2003, including Unforgiven, The Firm, Get Shorty and Wyatt Earp.
Goossen was a regular at his nephews baseball games. His nephew Josh Goossen-Brown was drafted by the Chicago White Sox in the 2014 MLB draft. Goossen was scheduled to be inducted into the Notre Dame High School Hall of Fame on February 26, 2011, but when he did not arrive for a photo session, a family member went to his nearby home in Sherman Oaks and found him dead from a stroke at the age of 65.