Born in Baltimore, Maryland, Davis grew up in East Baltimore. His father was a schoolteacher, his mother was a food services worker and his stepfather was a steel worker, according to an October 12, 2000 article in the Baltimore Sun. Davis attended Phillips Academy for high school, and earned a Bachelor of Arts degree in American history from the University of Pennsylvania in 1971. Although he had planned to become a college professor, Davis chose to pursue a career in the law after taking an undergraduate course in constitutional law, according to the October 12, 2000 article in the Baltimore Sun. Davis earned a Juris Doctor from the University of Maryland School of Law in 1978. At the University of Maryland, Davis won the Myerowitz Moot Court Competition in 1977 and was thus elected to the law school's three-member National Moot Court Team.
Prior to law school, Davis served as an assistant housing manager and equal opportunity specialist with the Housing Authority of Baltimore City. After graduating from law school, Davis clerked for United States District Judge Frank Kaufman of the United States District Court for the District of Maryland from 1978 until 1979. Davis then clerked from 1979 until 1980 for United States Court of Appeals Judge Francis Dominic Murnaghan Jr. From 1980 until 1981, Davis worked as an appellate attorney for the United States Department of Justice's Civil Rights Division. In 1981, Davis joined the United States Attorney's office for the District of Maryland as an Assistant United States Attorney until 1983, when he entered private practice. From 1984 until 1987, Davis worked as an assistant professor for the University of Maryland School of Law. He became a judge in 1987, when he was appointed to be an associate judge for the District Court of Maryland for Baltimore City. From 1990 until 1995, Davis worked as an associate judge for the Circuit Court for Baltimore City.
On May 4, 1995, President Bill Clinton nominated Davis to be a United States District Judge of the United States District Court for the District of Maryland. The United States Senate unanimously approved Davis to the seat in a voice vote on August 11, 1995. He received his commission on August 14, 1995. His service terminated on November 12, 2009, due to elevation to the Fourth Circuit. In discussing his judicial philosophy, Davis told the Baltimore Sun in an article that was published on October 12, 2000 that "I want the loser—and I know there's always going to be a loser, that's the nature of the beast—but I want the loser to be able to say, 'I lost, but I was heard, and I believe that judge gave me every consideration in hearing my side.'"
On October 12, 2000, President Clinton nominated Davis to be a judge for the United States Court of Appeals for the Fourth Circuit, to replace Murnaghan, who had died. The nomination was a part of Clinton's effort to integrate the Fourth Circuit, which up to that point had never had an African-American judge; however, since Davis was nominated after July 1, 2000, the unofficial start date of the Thurmond Rule during a presidential election year, no hearings were scheduled on his nomination, and the nomination was returned to Clinton at the end of his term. President George W. Bush chose not to renominate Davis to the Fourth Circuit.
On April 2, 2009, President Barack Obama nominated Davis to the United States Court of Appeals for the Fourth Circuit. On June 4, 2009, the Senate Judiciary Committee endorsed the nomination by a vote of 17-3. The full Senate voted 72-16 to confirm Davis on November 9, 2009. Davis received his commission on November 10, 2009. Davis assumed senior status on February 28, 2014, President Obama's first appointee to the federal courts to assume such status. On May 3, 2017, it was announced that Davis will become City Solicitor for the City of Baltimore, effective September 1, 2017. He retired from active federal judicial service on August 31, 2017.