Avery attended South Dade High School in Miami, Florida, from which he graduated in 1962.
He played college football at the University of Southern Mississippi, where he was a linebacker from 1964 to 1966. After playing on the freshman team for the 1963 season, he won the starting job at center in 1964 as a sophomore while also doing some double-duty playing defense for the 6-3 Eagles.
In 1965, he made the switch to full-time starting linebacker and led the team to a #1 ranking in the nation in total defense. He also still played some center and was a long-snapper for the 7-2 Eagles, with four of the wins shutouts.
In 1966, his senior season, the Eagles again led the nation in total defense for the 6-4 Eagles.
After his senior season he was named to the all-star Senior Bowl and Blue-Gray Games.
Avery was chosen in the 12th round of the 1967 NFL Draft by the National Football League's New York Giants and in the second round of the Red Shirt portion of the 1967 AFL Draft by the American Football League's Boston Patriots.
He signed with the Giants and played linebacker for the Giants in the 1967 and 1968 seasons during which he played in all but one game.
In 1969, he went to the AFL's Cincinnati Bengals, for whom he played six seasons from 1969 to 1974 (the team entered the NFL beginning with the 1970 season). In his second year with the team, 1970, he became a starting linebacker, starting 13 of their 14 games and recording his first career interception.
He remained an effective starter in 1971, again starting 13 games, then starting 12 in 1972. In 1973, he played in all 14 games, starting four, and recorded his second career interception. By 1974 he was no longer a starter, but did see action in 13 games.
His ninth and final pro season was in 1975 with the Kansas City Chiefs, for whom he played all 14 games, starting five.
He finished his career with the Kansas City Chiefs in 1975.
In 1983, he was inducted into the University of Southern Mississippi M-Club Alumni Association Sports Hall of Fame.
After retiring from football, Ken Avery founded a valet parking and self-parking company, American Parking Consultants. That led to him, in 1982, founding and serving as President of Ambassador Limousine in North Miami Beach, Florida. He is also the former President of the National Limousine Association.