A native of Reno, Nevada, Handley played three seasons at running back for Stanford University beginning in 1963. During his junior season, Handley gained 936 yards, with his long ground gains during the November 14, 1964 contest against Oregon State helping upset the Beavers, 16-7. The following year began with talk of potential All-American honors, but ended in disappointment, although Handley's running helped defeat arch-rival California-Berkeley, 9-7, in the season finale on November 21.
Handley was signed by the American Football League's San Diego Chargers on June 11, 1966, but voluntarily left camp one month later, putting an end to his playing career.
The following year, he returned to his alma mater as an assistant coach, then headed east in 1968 for two years as assistant at the U.S. Military Academy at West Point. One of his fellow coaches during this period was Bill Parcells, a connection that would prove fruitful in the future. Following a year out of the game in 1970, Handley returned the following year for the first of four more seasons at Stanford.
In 1975, Handley shifted to coaching another branch of the armed services when he accepted an assistant position at the United States Air Force Academy. In his final year, Parcells served as head coach, but after the latter's departure, Handley once again returned to Stanford, serving five years under both Rod Dowhower and Paul Wiggin.1967: Stanford University (GA)
1968-1969 United States Military Academy
1971-1974: Stanford University
1975-1978: Air Force Academy
1979-1983: Stanford University (LB)
On February 28, 1984, Handley was named offensive backfield coach of the New York Giants under Parcells, beginning a seven year stretch that saw the team capture two Super Bowls. In 1990, after coming close to leaving coaching to enter George Washington University Law School, Handley was promoted to offensive coordinator. Following the Giants' second title in Super Bowl XXV that season, Parcells announced his retirement, and Handley was then promoted to his first head coaching position on May 15, 1991.
George Young picked Handley to serve as head coach of the Giants after then-Giants defensive coordinator Bill Belichick left to coach the Cleveland Browns. The bad relationship between Young and Belichick also played into Handley's promotion; Young had consistently given a negative review about Belichick when teams with potential coaching opportunities inquired about him, and never felt his defensive coordinator had the leadership abilities to run his own team. In addition, Tom Coughlin, who Young had considerable respect for and would probably have promoted after Parcells' departure if the timing had worked, quickly moved after the Super Bowl and was hired as head coach at Boston College.
In 1991, Handley's first major decision of his tenure was who would be the starting quarterback. The previous season, Phil Simms had guided the Giants to an 11-2 record before suffering a leg injury. Backup Jeff Hostetler finished the regular season, and led the Giants to upset victories against the San Francisco 49ers in the NFC Championship game and the Buffalo Bills in the Super Bowl. Handley announced that the two quarterbacks would compete for the starting job. This caused some controversy as both fans and commentators felt that Simms shouldn't lose his starting job due to injury. Nonetheless, Hostetler won the battle in training camp and was announced as the starting quarterback.
Handley's first game as head coach was a Monday Night Football game against the San Francisco 49ers (a rematch of the 1990 NFC Championship). Handley's Giants would win that game, but Handley would be most remembered for wearing an unusually designed logo shirt (bearing the New York Giants colors and logo). The Monday night win helped Handley with most Giant fans, but when the team finished with an 8-8 record and out of the playoff picture, fan support quickly eroded. One fan held up a sign reading: "From the Super Bowl to the toilet bowl. Thanks, Ray." Fans also chanted "Ray must go!" during the 1991 and 92 seasons. In 1992, the team fell further to a 6-10 record, leading a combative Handley to trade verbal darts with both the media and his players.
His relationship with the defensive players was also strenuous, particularly when he tapped Rod Rust to be the defensive coordinator in 1992. The defense was not thrilled about Rust's hiring, especially knowing that the veteran coach was coming off a 1-15 mark as head coach of the New England Patriots in 1990. This led to some frustration during games, where the players would often ignore Rust's calls and call their own defensive assignments in the huddle. Handley's communication with the media became another reason for his dismissal, as he would often refuse to answer questions, even going as far as walking out one press conference after being asked about his handling of Hostetler and Simms as the team's starting quarterback. He scoffed at the question, calling it "ridiculous" and unfair to him and his quarterbacks. After the reporter persisted about why he wouldn't answer the question, Handley stormed out, telling the rest of the reporters to "get him straightened out."
Handley was officially fired on December 30, 1992, and replaced by former Denver Broncos head coach Dan Reeves. Following his departure, Handley would disappear from the NFL and the public eye. He now resides in the Lake Tahoe area. Before Super Bowl XLII, a reporter unearthed Handley's unlisted home telephone number and called him, in the hopes of talking about the 2007 Giants. Handley was unhappy at being contacted, and after being told it was for a "where are they now" interview, he said "No, I'm not the least bit interested. Thank you very much." and hung up the phone, making his lone post-firing public statement a very brief one.