McElroy was born in Providence, Rhode Island to Edward J. McElroy, Sr. and his wife Clara Angelone McElroy. He graduated with an A.B. degree in education from Providence College in 1962.
McElroy began a career as an educator teaching social studies and English at Lockwood Junior High School in Warwick, Rhode Island after graduation. He was elected president of the Warwick Teachers Union, AFT Local 915, for a two-year term in 1967. In 1969, he was elected president of the Rhode Island Federation of Teachers (RIFT). He continued to teach until 1972, when he became a full-time RIFT president.
As president of RIFT, McElroy oversaw rapid expansion in the union's membership. He also led the union in organizing school paraprofessionals, public employees, higher education faculty and campus workers, and nurses and other workers in hospitals and other health care organizations. McElroy stayed personally involved in the life of the union by handling negotiations and arbitrations for many of RIFT's education locals.
McElroy also was active in state politics. He served on the executive committee of the Rhode Island Democratic Committee, and several workforce development commissions and boards for the state.
In 1974, McElroy was elected a vice president of the AFT, and took a seat on the AFT's executive council. McElroy was instrumental in 1990 in launching the AFT's 'Futures Committee,' a panel of AFT vice presidents who spent two years studying and revamping the union's governance structures. The resulting constitutional amendments enhanced the role of the AFT's constituencies outside the preK-12 teacher division and made other recommendations on strategic planning, financial practices and priorities for the AFT.
In 1977, the 36-year-old McElroy was elected president of the Rhode Island AFL-CIO.
In 1992, the AFT's long-time secretary-treasurer, Robert Porter, died of a heart attack and McElroy was elected as his replacement. During his tenure as secretary-treasurer, McElroy strengthened the union's budgeting process and internal financial controls, and instituted financial training for all local affiliate treasurers.
In December 2001, McElroy was elected to the executive council of the AFL-CIO. He served on various AFL-CIO committees, including its political, education, strategic approaches, women workers and state and local strategies committees. After his election as AFT president, McElroy served on the AFL-CIO's Executive Committee as well.
In 2004, AFT president Sandra Feldman, suffering from a recurrence of breast cancer, resigned. McElroy was elected president at the union's biennial convention that July. McElroy was widely seen as a transitional president. He focused his tenure as AFT president on broad trade union issues, building the union's influence within the AFL-CIO, strengthening the AFT's state federations and middle- and small-size locals.
McElroy announced on February 12, 2008, that he would retire at the AFT's regularly schedule biennial convention in Chicago, Illinois, in July 2008. He was succeeded by New York City United Federation of Teachers President Randi Weingarten on July 14, 2008.
McElroy is a leader in a wide range of community, civic and labor organizations. He is a member of the board of the Albert Shanker Institute, and chair of the Department for Professional Employees, AFL-CIO (a coalition of 25 national unions representing 4 million professional, technical and administrative professionals).
McElroy also serves on the boards of the Amalgamated Bank of Chicago, Working America (an affiliate of the AFL-CIO) and the National Labor College (formerly the George Meany Center for Labor Studies).
McElroy served as a member of the Debt Reduction Task Force at the Bipartisan Policy Center.
McElroy, a Roman Catholic, resides in Washington, D.C. He is married to Edwina B. (Ricci) McElroy; they have four children.