Baldwin was born at North Woburn, Massachusetts living at Baldwin House aka "The Baldwin Mansion", educated at Westford Academy, and graduated from Harvard College in 1800. His early inclinations were towards mechanical subjects, and during his college life he made a clock which kept good time.
In 1794 at age 14 he accompanied his father and two brothers (Cyrus, 22, and Benjamin, 17) on a nine-day consulting inspection visit from the famous canal engineer William Weston of the route of the Middlesex Canal. Later all five of Loammi Baldwin Senior's sons ( Loammi Baldwin Jr., Benjamin Franklin Baldwin, Cyrus Baldwin, James Fowle Baldwin and George Rumford Baldwin) worked with their father on the ten-year construction of the Middlesex Canal.
After graduation, he entered a law office in Groton, Massachusetts, where he constructed a fire engine called "Torrent", which the town greatly needed. In 1804 he completed his studies at Groton and opened his own law office in Cambridge. In 1807, however, he abandoned his law practice in favor of engineering. He traveled to England to inspect public works and intended to do the same in France, but he was barred by the difficulty in entering the country. Upon his return he began an engineering practice in Charlestown, Massachusetts. He was elected a Fellow of the American Academy of Arts and Sciences in 1810, and a member of the American Antiquarian Society in 1814.
One of his earliest engineering works was the construction of Fort Strong (1814), a fort in Boston Harbor built for defense against the British. He served as chief engineer with the rank of Colonel, a title which has sometimes confused him with his father. In 1819 he was asked to complete the construction of the Milldam, now that stretch of Beacon Street beyond the Boston Common. From 1817 to 1820, he worked in Virginia, and in 1821 was made engineer of the Union Canal in Pennsylvania. In 1824 Baldwin returned to Europe and remained there a year, mostly in France, examining public works.
In 1825 he joined a small committee planning the Bunker Hill Monument and recommended the obelisk now seen there. In 1827 he accepted an appointment from the United States government which led to the two great works of his life (1827-1834): the naval dry docks at the Boston Navy Yard in Charlestown and at the Norfolk Naval Shipyard in Portsmouth. Baldwin led many other projects such as a marine railway at Pensacola; construction of buildings at Harvard College; a canal around the Ohio River falls; a stone bridge called the Warren Bridge at Charlestown; and the Harrisburg Canal in Pennsylvania.
Mr. Baldwin was twice married; first to Ann, daughter of George Williams, of Salem. She was sister of Samuel Williams, an eminent American banker in London; second on June 22, 1828, to Catherine, widow of Captain Thomas Beckford, of Charlestown. She died May 3, 1864. Child by first marriage: Samuel Williams Baldwin, born 1817, died December 28, 1822.
About a year before he died he had a stroke of paralysis; a second attack proved fatal. He died at Charlestown, Massachusetts on June 30, 1838, at the age of fifty-eight, and was buried in Woburn. After Loammi's stroke and death his brother, James Fowle Baldwin (1782-1862), with whom he had often worked, continued his work especially for a safe water supply for Boston.