Chaney was born in Jacksonville, Florida. He began his career after graduating from Bethune–Cookman College and spending some time in the Eastern Professional Basketball League, first with the Sunbury Mercuries from 1955 to 1963 and Williamsport Billies from 1963 to 1966.
Chaney first became a basketball coach in 1963 at Sayre Junior High School and went 59–9 in three seasons. Inheriting a one-win team in 1966 at Simon Gratz High School in Philadelphia, Chaney compiled a 63-23 record in six seasons.
The first collegiate position held by Chaney was at Division II Cheyney State. At Cheyney, Chaney was 232-56. He won a national title in 1978.
After a decade at Cheyney, Chaney moved on to Division I Temple in Philadelphia. Chaney built a reputation as a tough coach who always demanded excellence on and off the court. He was well known for his early-morning practices, match-up zone defense, tough non-conference scheduling, and winning basketball teams.
Chaney won a total of 741 career games. He took Temple to the NCAA tournament 17 times. His 1987-88 Owls team entered the NCAA tournament ranked #1 in the country, and he reached the Elite Eight on five different occasions.
In 2001, Chaney was elected to the Naismith Memorial Basketball Hall of Fame.
On December 20, 2004, during a win over Princeton, Chaney became the fifth active coach and 19th all-time to appear on the sidelines for 1,000 games, joining Lou Henson (New Mexico State, Illinois), Bob Knight (Army, Indiana, Texas Tech), Eddie Sutton (Creighton, Arkansas, Kentucky, Oklahoma State, San Francisco), and Hugh Durham (Florida State, Georgia, Jacksonville).
On March 13, 2006, Chaney announced his retirement from coaching at a press conference, to be effective after Temple's play in the National Invitation Tournament (NIT). Fran Dunphy was named Chaney's successor following the season. Chaney has since been inducted into the Big 5 Hall of Fame, which recognizes the best in Philadelphia's college basketball history.
On February 13, 1994, controversy ensued when he threatened to kill then-University of Massachusetts Amherst coach John Calipari at a post-game news conference, where Calipari was speaking at a podium.
Chaney entered the conference mid-speech, accusing Calipari of manipulating the referees. When Calipari attempted to respond to the accusations, Chaney yelled, “Shut up goddammit!”, and proceeded to charge the stage, before being stopped by security. While being held back, Chaney shouted, "When I see you, I'm gonna kick your ass!". As security restrained Chaney, he repeatedly yelled, "I'll kill you!" and angrily admitted telling his players to "knock your fucking kids in the mouth." Chaney received a one-game suspension for the incident. Chaney has recently praised Calipari's coaching ability and defended him over the Derrick Rose controversy at the University of Memphis.
Chaney made headlines in 2005 after ordering backup forward Nehemiah Ingram into the game to commit hard fouls against Big 5 rival Saint Joseph's in response to what he thought were several missed calls by the referees. After the game Chaney admitted to "sending a message" and stated "I'm going to send in what we used to do years ago, send in the goons." John Bryant of Saint Joseph's suffered a fractured arm as a result of an intentional foul. Following the incident, he suspended himself for one game, and upon hearing the severity of the injury, the university suspended him for the remainder of the regular season. Temple then later extended the suspension to the Atlantic 10 tournament. He returned for a farewell season that ended in a loss to Saint Joseph's in the A-10 Tournament.Chaney has led teams to an overall record of 741-312 and 31 post season berths
Compiled a 225-59 record at Cheyney State
NCAA Division II Tournament, 1973, 1974, 1976–80, 1982
NCAA Division II Tournament National Champions, 1978
Division II National Coach of the Year, 1978
State of Pennsylvania Distinguished Faculty Award, 1979
Compiled a 516-253 record at Temple
Compiled a 296-100 Atlantic 10 Conference Regular Season Record
Won his 400th game at Temple against No. 1 ranked Cincinnati on Feb. 20, 2000
NCAA Tournaments, 1984–1988, 1990-2001 (17)
NCAA Regional Finalists, 1988, 1991, 1993, 1999, 2001
National Invitational Tournament, 1989, 2002, 2003, 2004, 2005,2006
Atlantic 10 regular season Championships, 1985, 1987, 1988, 1990, 1998, 1999, 2000, 2002
Atlantic 10 Tournament Championships, 1985, 1987, 1988, 1990, 2000, 2001
Atlantic 10 Conference Coach of the Year, 1984, 1985, 1987, 1988, 2000
Has compiled 15 20-win seasons at Temple
USBWA National Coach of the Year, 1987, 1988
Consensus National Coach of the Year, Associated Press Coach of the Year
Eastern Basketball Coach of the Year, 1993
Won his 700th game, becoming the first African-American in history with 700 wins