He was born in New York City, the only son of Robert Lenox, a wealthy Scottish merchant of New York, from whom he inherited a fortune of several million dollars and 30 acres of land between Fourth and Fifth Avenues in 1839. A graduate of Columbia College, he studied law and was admitted to the bar, but never practiced. He retired from business when his father died.
He went to Europe soon after his admission to the bar, and while abroad began collecting rare books, which, along with collecting art, became the absorbing passion of his life. For half a century, he devoted the greater part of his time and talent to forming a library and gallery of paintings not surpassed in value by any private collection in the New World. These, together with many rare manuscripts, marble busts and statues, mosaics, engravings, and curios, became the Lenox Library in 1870 in New York City.
The library occupied the crest of the hill on Fifth Avenue, between 70th and 71st Streets, overlooking Central Park. On 23 May 1895, the Lenox Library was consolidated with the Astor Library and the Tilden Trust to form the New York Public Library.
The collection of Bibles, including the Mazarin, both as to number and rarity, was believed to be unequalled even by those in the British Museum, while its Americana, incunabula, and Shakespeariana surpassed those of any other American library, public or private. The collection was valued at nearly a million dollars, which, with the $900,000 for the land and building and the endowment, made a total of above $2,000,000. The Frick Collection stands on the Lenox Library's former site.
Lenox was a founder of the Presbyterian Hospital in New York City, his gifts to it amounting to $600,000. He also made important gifts to Princeton College and Seminary, and gave liberally to numerous churches and charities connected with the Presbyterian Church. Lenox was also the president of the American Bible Society, to which he was a liberal donor. James Grant Wilson reports passing on several gifts to needy men of letters accompanied by the condition that Lenox should not be known as the donor. He was elected a member of the American Antiquarian Society in 1854, and served as the society's vice-president from 1868 to 1880.
He never married. A love in early life, the only lady to whom Lenox was ever attached, refused him and remained single even after his death. This event increased his peculiarly reserved and retired habits, and he became and continued a recluse, never being seen in the New York society to which by birth and connection he belonged. He declined proffered visits from the most distinguished men of the Old World and the New. An eminent scholar, who was occupied for many weeks in consulting rare books not to be found elsewhere, failed to obtain access to the library of Lenox. He was assigned an apartment in Lenox's spacious mansion for his use, and to that apartment the works were sent in installments without his ever penetrating into the hall containing the precious collection, or to the presence of its possessor.
Lenox occasionally reprinted limited editions, restricted to ten or twenty copies, of rare books, which he placed in some of the great public libraries and notable private collections, for example that of John Carter Brown. Portraits of Lenox were painted by Francis Grant in 1848, and by G. P. A. Healy three years later. He was also painted by Daniel Huntington in 1874.
In 1855 there were 19 millionaires in New York. He was the third richest man in New York worth approximately 3 million dollars.
He is buried in the New York City Marble Cemetery. Two of his seven sisters outlived him. Henrietta Lenox, the last survivor, gave the Lenox Library 22 valuable adjoining lots and $100,000 for the purchase of books. Lenox Avenue in Harlem is named for James Lenox.