After graduating from Lash High School in Zanesville in 1943, Taylor entered the United States Army Air Forces where he served from 1943 to 1946. Despite never having played high school basketball, he became an outstanding player at Ohio State and was the starting forward on the 1950 Big Ten Conference championship basketball team. Taylor did learn the game of basketball while in the Army Air Forces playing under Captain Rowland Wenzel going undefeated. In addition he was Ohio State University's first All-American baseball player. His number 27 is now retired at Ohio State.
After graduating, Taylor signed as an amateur free agent with the baseball Washington Senators on June 6, 1950. Primarily a first baseman, he was assigned to the minor league Chattanooga Lookouts, where he batted .263 in 78 games. He made his major league debut on September 12, and played six games for the Senators.
Taylor returned to Chattanooga in 1951, this time batting .291 in 152 games. He again earned a brief trial with the Senators, appearing in six more games. In 1952, he managed to play in ten games while again spending most of the season in the minors. After playing one more season with the independent Beaumont Explorers of the Texas League in 1953, he left baseball for good.
After the end of his baseball career, Taylor returned to Ohio State as assistant basketball coach in 1958, becoming head coach the following year.
During his 18 years at Ohio State, the Buckeyes won the 1960 NCAA championship, were finalists in 1961 and 1962 and claimed a third-place finish in 1968. The last time he coached the Buckeyes to an NCAA tournament appearance was in 1971, where OSU upset previously unbeaten Marquette in the Mideast regional semifinal round. However, Western Kentucky beat OSU in the Mideast regional round to advance to the Final Four. In his five NCAA tournament appearances, Taylor's teams went 14–4 and also won or shared seven Big Ten titles.
Taylor finished his career with an overall record of 297–158 and was named Coach of the Year by the USBWA and UPI in 1961 and 1962. A talented recruiter, Taylor coached six All-Americans as well as Hall of Famers Jerry Lucas, John Havlicek and Bobby Knight.
Taylor served as President of the National Association of Basketball Coaches in 1972, was a member of the U.S. Olympic Basketball Committee from 1964 to 1972
After retiring from coaching in 1976, Taylor managed the U.S. National Team in the 1978 FIBA World Championships and the 1979 Pan American Games. In addition, Taylor managed The Golf Club, a private golf course in New Albany, Ohio for 18 years.
In addition, Taylor was a television analyst for college basketball on NBC, often paired with Merle Harmon, during the late 1970s and early 1980s. As players found themselves in precarious situations, Taylor often described them as, "between a rock and a hard place."
On May 6, 1986 Taylor was enshrined in the Basketball Hall of Fame, having previously been selected to the Ohio State Hall of Fame. He was inducted into the Ohio Basketball Hall of Fame as part of the Charter Class of 2006.