Clinton was born in Ulster County in the colony of New York, at Little Britain in the town of New Windsor, now part of Orange County, New York. He was the third son of Col. Charles Clinton, an Anglo-Irish colonist and a colonel in the French and Indian War who emigrated to New Ulster in 1729, and Elizabeth Denniston.
He was the brother of George Clinton, Governor of New York from 1777 to 1795 and U.S. Vice President from 1805 to 1812, the grandson of James Clinton (d. 1718), and the great-grandson of William Clinton (1614-1684), a royalist officer in the army of Charles I of England.
James Clinton's military experience began in the French and Indian War, where he served in the New York militia. He was commissioned an ensign in 1757 and achieved the rank of captain in 1759. In 1758, commanding a company, he participated, along with his father (Colonel) and brother George (Lieutenant), in General John Bradstreet’s capture of Fort Frontenac (now Kingston, Ontario). He and his brother played a key role in capturing a French vessel.
James remained in the army, stationed at various frontier posts. In 1763 he raised and commanded a corps of two hundred men, who were designated as "Guards of the Frontier". After the war he retired and married Mary De Witt.
A month after the first open armed conflict in Lexington, the Continental Congress resolved on May 25, 1775 to build fortifications in the Hudson highlands for the purpose of protecting and maintaining control of the Hudson River. James Clinton and Christopher Tappan, lifetime residents of the area, were sent to scout appropriate locations. Clinton was commissioned as a colonel of the 3rd New York Regiment, which took part in Brig. Gen. Richard Montgomery’s unsuccessful expedition to Quebec in 1775. In March 1776, Clinton took command of the 2nd New York Regiment and soon after, in August, was promoted to brigadier general in the Continental Army.
He served most of the war in the Northern Department, along the New York frontier. During the Saratoga Campaign in 1777, he commanded Fort Clinton in the Hudson Highlands. He participated in a successful effort to prevent British General Sir Henry Clinton from rescuing General John Burgoyne at Saratoga, but he and his troops were unable to hold Forts Clinton and Montgomery. Clinton sustained a bayonet wound in the leg during the assault. In 1778 he was stationed in Albany to oppose Indian and Tory forces.
In 1779, Clinton led an expedition down the Susquehanna River after making the upper portion navigable by damming up the river's source at Otsego Lake, allowing the lake's level to rise, and then destroying the dam and flooding the river for miles downstream. This event is described by James Fenimore Cooper in the introduction to his popular novel The Pioneers, and commemorated by a Memorial Day canoe race.
At Tioga, New York, Clinton met up with General John Sullivan's forces, who had marched from Easton, Pennsylvania. Together, on August 29, they defeated the Tories and Indians at the Battle of Newtown (near today's city of Elmira, New York). This became known as the "Sullivan-Clinton Campaign" or the "Sullivan Expedition."
In 1780, Clinton temporarily commanded the Northern Department. By October 1781, his brigade had joined George Washington's army in the siege of Yorktown.
After he left the army, Clinton served on the commission defining the New York-Pennsylvania boundary. In 1783 General Clinton became an original member of the New York Society of the Cincinnati. He also served as an assemblyman in the New York State legislature from 1787-1788 and again from 1800-1801, and as a New York State Senator from 1788-1792.
On February 18, 1765, James Clinton married his first wife, Mary DeWitt (1737–1795), the only daughter of Egbert DeWitt, members of an old Dutch family. They had seven children, including:Alexander Clinton (1765–1787), who served in Colonel Lamb's regiment during the Revolution and drowned in the Hudson river
Charles Clinton (1767–1829), who married Elizabeth Mulliner (1770–1865)
DeWitt Clinton (1769–1828), later also a Governor of New York
George Clinton, Jr. (1771–1809), who served in Congress
Mary Clinton (1773–1808), who married Robert Burrage Norton. After his death, she married Judge Ambrose Spencer (1765–1848)
Elizabeth Clinton (1776–1832), who married William Stuart
Katharine Clinton (1778–1837), who married Samuel Lake Norton, brother to her sister Mary's husband. After his death, she married Ambrose Spencer, her sister's widower
His second wife was Mary (née Little) Gray (1768–1835), the widow of Alexander Gray (1762–1795) who was born in Ireland. Together, James and Mary were the parents of six children:James G. Clinton, who died young.
Caroline Hannah Clinton (1800–1864), who married Charles Augustus Dewey (1793–1866), an Associate Justice of the Massachusetts Supreme Judicial Court
Emma Little Clinton (1802–1823), who died unmarried
James Graham Clinton (1804–1849), who married Margaret Ellsworth Conger and served in Congress.
Letitia Clinton (1806–1842), who married Dr. Francis Bolton (1804–1849)
Anna Clinton (1809–1833), who married Lt. Edward Ross
Clinton died in Little Britain, New York, on September 22, 1812, the same year as his brother George.
Through his son DeWitt, he was the grandfather of ten, including George William Clinton (1807–1885) who served as Mayor of Buffalo, New York from 1842 to 1843. Through his son George, he was the grandfather of three.