Born in Altoona, Pennsylvania, Loy earned the rank of Eagle Scout in the Boy Scouts of America as a youth and was awarded the Distinguished Eagle Scout Award as an adult. Loy entered the United States Coast Guard Academy in 1960. Subsequently, he earned master's degrees in history and government from Wesleyan University, and in Public Administration from University of Rhode Island.
Loy served as a commissioned officer in the U.S. Coast Guard, served in combat as commanding officer of a patrol boat in the Vietnam War, and eventually rose to the rank of admiral. In May 1998, Loy became the twenty first Commandant of the Coast Guard, serving in that post until 2002.
As the USCG Commandant, Loy reacted to the September 11 attacks of 2001. In the short term, he supervised the resumption of sea-borne trade throughout the U.S., after the USCG had shut down most major ports after the attacks. In the long term, Loy led the U.S. delegation to the International Maritime Organization (IMO), and was instrumental in ensuring that the International Ship and Port Facility Security Code was approved and implemented in 2002. The code came into effect in 2004.
In May 2002, the Secretary of the U.S. Department of Transportation, Norman Mineta, appointed Loy to become the Deputy Undersecretary for the newly formed Transportation Security Administration. Loy led the agency through its creation and subsequent incorporation into the Department of Homeland Security.
On October 23, 2003, Loy was nominated as the U.S. Deputy Secretary of Homeland Security by U.S. President George W. Bush, and sworn in on December 4, 2003. Following the departure of Tom Ridge, Loy filled in as Acting Secretary of Homeland Security from February 1, 2005, until February 15, 2005, when Michael Chertoff was confirmed and sworn into office. Joining the exodus of leadership, Loy resigned as Deputy Secretary, effective March 1, 2005.
On April 7, 2005, the Cohen Group announced that Loy had joined the firm as a Senior Counselor, effective April 18.
On August 5, 2005, Loy joined the Board of Directors for Lockheed Martin.
In the fall of 2006 it was announced that Loy was being honored as the first Chair of the Tyler Institute for Leadership at the U.S. Coast Guard Academy. As such Loy has played a significant role in speaking and drawing other distinguished visitors to the Academy. His first class was designed to teach a select group of cadets about the international shipping industry and how it might be secured.
In March 2007, the Washington Post had a feature on Loy and his relation to the U.S. Coast Guard's Deepwater contract, which was awarded to Lockheed Martin in summer of 2002. When asked by the Washington Post if he ever faced improper influence on Deepwater decisions while serving as the USCG Commandant, Loy said: "The question is almost insulting. I will pass on giving you any kind of answer." Admiral Loy is a recipient of the Distinguished Eagle Scout Award.