McIlvaine was born in 1799 in Burlington, New Jersey to Joseph McIlvaine (later United States Senator from New Jersey) and Maria Reed (daughter of Bowes Reed, the Secretary of State of New Jersey, and niece of Joseph Reed, Continental Congressman and Governor of Pennsylvania). His father was of Scottish origin, from the MacIlvaines of Ayrshire.
McIlvaine was educated at Burlington Academy and entered the College of New Jersey (later Princeton University), where he graduated in 1816. The following year, he entered the theological seminary attached to the First Presbyterian Church of Princeton.
In 1820 he was ordained to the diaconate in Philadelphia, and was soon after called to Christ Church in Georgetown, Washington, D.C. In 1822 he was appointed chaplain to the U.S. Senate.
From 1825 to 1827, McIlvaine served as chaplain and professor of ethics at the United States Military Academy at West Point, where his students included Robert E. Lee and Jefferson Davis.
In 1827 McIlvaine declined the presidency of The College of William & Mary but accepted a call to St. Ann's Church in Brooklyn, New York. In 1831 he was named professor of the evidences of revealed religion at the University of the City of New York.
In 1832, he became the 2nd president of Kenyon College in Gambier, Ohio, and also the second Bishop of Ohio.
He was a leading advocate of Evangelicalism, and wrote a noted rebuttal of the Oxford Movement, Oxford Divinity Compared with That of the Romish and Anglican Churches.
He was the 28th bishop consecrated in The Episcopal Church.
Bishop McIlvaine was so highly respected internationally (for his opposition to the Catholic-leaning Oxford movement within the Episcopal Church) that, shortly after the outbreak of the Civil War, President Lincoln asked him to go to England to argue against British recognition of the Confederacy. He often had coffee at Buckingham Palace, lunched with faculty members at Oxford, conversed with cabinet members, and influenced debate in the House of Commons.
McIlvaine died in Italy in 1873. His body, carried through England on its journey home to Ohio, was honored for four days in Westminster Abbey, the only American to this day to lie-in-state at Westminster.