Parker was the son of Ernest and Mabel Parker and grew up near Norfolk, Virginia. He attended Woodrow Wilson High School in Portsmouth, graduating with the class of 1933 and starring in five sports. He enrolled at Duke University as a freshman in 1933.
At Duke, Parker competed in three sports: football, basketball and baseball. From 1934–1936, he starred at running back, doing most of the running and passing for Duke. He was second team All-American in 1935 and consensus All-American first team in 1936. He placed sixth in the Heisman Trophy voting in 1936. Parker was a great open-field runner and one of the best punters in college football at the time. His 105-yard kickoff return against North Carolina is still a Duke school record. Parker also stood out as a baseball player at Duke, playing in 1935–1936.
In his senior season at Duke, he served as team captain for the Duke Blue Devils who went 9–1, captured the league title with a 7–0 record, and finished the season ranked 11th in the Associated Press national poll.
He was elected into the College Football Hall of Fame in 1955. He was inducted into the North Carolina Sports Hall of Fame in 1963, the Virginia Sports Hall of Fame in 1972, and was an inaugural member of the Duke University Sports Hall of Fame, inducted in 1975.
Parker was drafted by the Brooklyn Dodgers as the third pick of the second round in the 1937 NFL draft. Sammy Baugh was the only passer drafted ahead of Parker. Parker, who played for the Philadelphia Athletics of Major League Baseball beginning in 1937, originally had no intention of playing in the NFL. Baseball was the glamour pro sport at the time and the NFL had a rough, vulgar reputation. But perhaps because of his .117 batting average that year, he asked for and received permission from the A's to play football. Parker thus became a true two-sport phenomenon, playing both Major League Baseball and NFL football in both 1937 and 1938. Parker, playing various infield positions, batted .179 over two seasons with the A's, scoring 20 runs with 25 RBI over 94 games. Parker was the first of only seven Major League Baseball players to hit a home run as a pinch-hitter in their first at bat.
When Parker joined the Dodgers in 1937, Brooklyn had been a perennial NFL cellar-dweller in the East Conference since 1930. With his running, passing, and punting ability, he brought them instant credibility. He led the team in passing in 1937 and every year he played. In 1938, he led Brooklyn to a 0.500 record and led the NFL in passing yards with 865. When legendary coach Jock Sutherland joined the Dodgers in 1940, Parker's career took off. In 1940, he threw for 817 yards and 10 TDs, rushed for 306 yards, caught 3 passes, including 2 for TDs, and led the league in points after touchdowns. The Dodgers finished only one game out of first, with an 8–3 record, and Parker was named the NFL MVP. In 1941, Parker continued to shine, but the Dodgers again finished second to the New York Giants, despite beating their New York rivals twice during the season. Parker's NFL career went on hold in 1942, as he, like many NFL players, left football to enlist in the Armed Services. After serving for over two years, Parker returned to the NFL, this time with the short-lived Boston Yanks, but at age 33, he took on a minor role.
He rejoined the former owner of the Dodgers, Dan Topping, in 1946 as part of the New York Yankees of the new All-America Football Conference (AAFC). Coached by former Washington Redskins coach Ray Flaherty and led by Parker, the Yankees won the AAFC East, giving Parker his only division title in pro football. The Yankees met the powerful Cleveland Browns in the championship game. The Yankees played well, but eventually succumbed to the Browns. Parker was 8 of 18 passing, for only 81 yards and an interception. Parker retired after the game, completing a fine career at age 34. He was inducted into the Pro Football Hall of Fame and the Virginia Sports Hall of Fame in 1972.
After his playing days, Parker became the head baseball coach (1953–1966) and assistant football coach (1947–1965) at Duke University. He was manager of the Durham Bulls from 1949 to 1952, serving as player-manager for the first three seasons and finishing with a record of 303–266 (.533). He was Piedmont League manager of the year in 1949 and 1951. He was also a founding member of the Elizabeth Manor Golf and Country Club in Portsmouth, Virginia.
On August 13, 2008, Parker was part of the inaugural class inducted into the Hampton Roads Sports Hall of Fame, honoring athletes, coaches and administrators who made contributions to sports in Southeastern Virginia.
At the time of his death, Parker was the oldest living member of the Pro Football Hall of Fame, the oldest living former professional football player and the last living person to play on the same major league baseball field as Baseball Hall of Fame member Rogers Hornsby. On May 7, 1937, Parker appeared for the Philadelphia Athletics while Hornsby played one of his last games for the St. Louis Browns. Before his death, Parker and Hall of Famer Bobby Doerr were the last men to play on the same field as baseball immortal Lou Gehrig.
Parker died the morning of November 6, 2013 at the age of 101. He is the first and so far only member of the Pro Football Hall of Fame to have lived past their hundredth birthday.