Born in Philadelphia, Pennsylvania, Patterson grew up in Miami, Florida. He was a highly touted prospect until a devastating right shoulder injury cut his promising playing career short. Nevertheless, he developed into one of the most respected Minor League pitching coaches and instructors in baseball.
The Yankees selected Patterson in the first round (7th pick) of the 1975 MLB Draft out of Miami Dade Community College in Miami, FL. A 19-year-old rookie, he was immediately assigned to Class Low-A Oneonta Yankees, where he posted an 8–4 record with a 1.95 ERA and one shutout in 14 games (13 starts), while his 97 strikeouts in 106.0 innings pitched led the New York-Penn League. The Cincinnati Reds tried to trade for him, offering Tony Pérez, a future Hall of Famer, but the Yankees declined the offer.
In 1976, Patterson was impressive, going 9–2 with a 2.44 ERA at Double-A West Haven Yankees, and throwing the only no-hitter of the Eastern League season on June 28 against the Williamsport Tomahawks. He gained a promotion to Triple-A Syracuse Chiefs, where he was 7–2 with a 2.92 ERA. Overall, he went 16–4 with a 2.44 ERA in the two stints, including two shutouts and 12 complete games over 23 starts in 177.0 innings. A while later, he played winter ball with the Cardenales de Lara club of the Venezuelan League in its 1976-77 season while being managed by Bobby Cox.
In 10 starts, Patterson went 5–3 with a 4.56 ERA in 71.0 innings, totalling 248.0 innings in less of a year, a significant work overload for a young pitcher that eventually took a toll on his throwing arm. On the other hand, Yankees general manager Gabe Paul ordered Patterson to play winter ball in Venezuela, neglecting his workload of nearly 500 innings in a span of two seasons and Patterson, at 21, did as he was told.
Patterson opened 1977 at Syracuse before joining the Yankees in the month of April. He soon felt a popping sensation in his arm and fooled people for a while, but deep down and, according his own words, he knew he would never be the same. Patterson would only pitch in ten games for New York in the season, ending with a record of 1–2 and a 5.40 ERA. He then was done in the majors due to arm pain.
Afterwards, Patterson had eight operations and was unable to pitch for over two years. While recovering, he started the process of teaching himself to pitch left-handed to return to professional baseball. Additionally, he continued to try to get the right arm ready to go. At the beginning, Patterson threw the ball hard, but the ball would not even bounced back and had relapses in the process. Eventually, he received advice from pitching legend Whitey Ford and pitched in a spring training game left-handed. He was fine, but continued to throw right handed in the minor leagues during three seasons.
His comeback started on a high note, as he was 5–2 with a 1.58 ERA in eight starts for Rookie Class Gulf Coast League Yankees in 1980, finishing fourth in the league in ERA. In 1981, he was 1–4 with a 3.20 ERA for Class-A Fort Lauderdale Yankees, and then 1–1, 1.33 ERA for Class-A Fresno Giants in 1982 to complete his revival bid, but did not return to the majors. He compiled a 33–16 record with a 2.37 ERA in 63 minor league appearances (62 starts).
Patterson was out of baseball by 1983, though he held out hope for a comeback. He took a job parking cars at a restaurant in Fort Lauderdale, Florida, in which he threw left-handed against the outside walls after closing. The Yankees made their spring training in town, and its owner George Steinbrenner went to the restaurant one night and recognized Patterson. Steinbrenner offered him a coaching job, but it did not last long. As a result, Patterson began his coaching career in 1984 with his first team, the Oneonta Yankees, but would not let history repeat. He then was dismissed after the season for refusing to let the then 19-year-old prospect Al Leiter pitch with a sore arm.
In 1991, Patterson joined the Oakland Athletics organization, where he served as the pitching coach for the Madison Muskies from 1992 to 1993 and the West Michigan Whitecaps in 1994. Then in 1996, he was the roving minor league instructor. In between, when arm problems jeopardized Leiter's career, Patterson rebuilt the delivery of his longtime friend and neighbor, as Leiter rejuvenated his career and kept pitching until he was 40 years old.
Patterson later worked in the Arizona Diamondbacks system as their minor league pitching coordinator from 1997 to 2000. After that, he got a job with the Toronto Blue Jays as their bullpen coach in 2001 and then as their pitching coach from 2002 through 2004.
In 2005, Patterson rejoined the Yankees and worked as pitching coach for Triple-A Columbus Clippers, being reassigned to the GCL Yankees from 2006 to 2007. Patterson had asked for the assignment to work in the Gulf Coast minor league farm team, where there are no overnight trips and the schedule allowed him to be home by the late afternoon. At that time, Patterson decided he had to stay home to be close to his 7-year-old son, who had been diagnosed with Tourette syndrome, a condition of the nervous system characterized by involuntary, uncontrolled movements and vocalizations called tics.
From 2008 to 2012, Patterson served as the minor league pitching coordinator for the Athletics. He took over the same post with the Yankees from 2013 to 2015, and came back to Oakland in 2016 for his third stint in the organization as the head of the Athletics' minor league pitching program. Pitching development has been the backbone of the Athletics since the franchise moved to Oakland, while Patterson is one of the men who helped to build that program in the early 1990s.