Little is known about Rumsey until he was living in Bath, Virginia, (now Berkeley Springs, West Virginia) in 1782. He likely had moved to the area with his family some years before the American Revolution, from Cecil County, Maryland, where he had helped to run the family water mill at Bohemia Manor. His cousin was Benjamin Rumsey, a notable Maryland jurist and statesman, who also grew up at Bohemia Manor. In Bath, he built houses, became a partner in a mercantile business, and helped to run a boarding house and tavern called the "Sign of the Liberty Pole and Flag."
In September 1784, when George Washington was staying at Rumsey's inn, he contracted with Rumsey to build a house and stable on property he owned at Bath. During this visit, Rumsey showed Washington a working model of a mechanical boat which he had designed. It had a bow-mounted paddlewheel that worked poles to pull the boat upstream. Washington had been making plans for making the Potomac river navigable since before the Revolution, and a company was soon to be formed for the purpose. Rumsey's pole-boat, which promised to be able to ascend the river's chutes and swift currents, must have seemed a godsend to Washington, who wrote a certificate of commendation for Rumsey and likely let him know of the river project. Armed with the certificate, Rumsey obtained a patent from the Virginia legislature for "the use of mechanical boats of his model" and also gained an investor.
In July 1785, he was recommended by Washington and appointed to go to the oversee the clearing of rocks at what is now Harper's Ferry Patowmack Company. Rumsey would thus not only be able to build a boat to ascend the river, but alter the river to enable his boat. For a year, Rumsey oversaw work on the Potomac River site, while his assistant and brother-in-law Joseph Barnes did much of the building of the boat around Shepherdstown.
Rumsey had quickly concluded that the pole-boat design was too limited and decided to incorporate steam propulsion into his design. While making his boat much more useful, it made it far more complex and expensive to build. It soon became was obvious that the Patowmack Company had a much greater task than any of its members had foreseen. It was hindered by the lack of an overall supervising engineer; overseers were having to improvise as best they could. The work required much manual labor and difficult blasting, and Rumsey found himself directing a large and restive gang of about a hundred workmen, including leased slaves and bondsmen, encamped in a remote area, without adequate supplies.
After a year Rumsey said he would resign if not given an increase in pay. His resignation was accepted and his assistant, Richardson Stewart, was given his job. Other aspects of the matter are open to debate; Stewart may or may not have worked against Rumsey to gain his job; Rumsey thought he had, the Company (and Washington) thought Rumsey's allegations unfounded. Still, according to Company minutes, Stewart was fired soon afterward, for "sundry charges of a serious nature".
Work on a hull had begun in 1785 in Bath by Joseph Barnes. The boat was brought that fall to Shepherdstown. Valve castings, cylinders, and other pieces which had been made in Baltimore and Frederick were installed that December, and the boat was taken downriver to Shenandoah Falls for a test. However, bad weather postponed testing until the following spring. When Rumsey finally tested the boat, it proved very unsatisfactory. The pole-boat mechanism caused the boat to yaw in the current, which disabled the paddlewheel and stopped the boat. In the steam pump, the engine consumed too much steam; the boiler was inadequate.
At some point in 1786, work on the pole-boat mechanism was abandoned. For a better boiler, he tried a coil of forged iron pipe, which proved to be both much more efficient and much smaller and lighter. With a functioning steam engine, another problem arose. The single-cylinder pump would draw several gallons of water from beneath the boat and send it down a copper pipe to the stern. Because gallons of water were being drawn into the pump at the same time as water was still flowing from it to the stern, the pump was working against itself; several strong strokes and it bound up. This was resolved by replacing the copper pipe with a square wooden trunk with flapper valves in the bottom to allow water in from the river, to relieve the negative pressure at the pump.
On December 3, 1787, the boat finally made a very successful public demonstration on the Potomac at Shepherdstown.
The demonstration took place twenty years before Robert Fulton constructed and demonstrated the Clermont. The idea of jet propulsion was not Rumsey's alone. Daniel Bernoulli (1700–1782) originated the idea of propelling watercraft in that way. In the summer of 1785, while Rumsey and his assistant Joseph Barnes were in the process of assembling his boat, Benjamin Franklin, on board a ship from France, wrote of propelling a boat by water jet. This coincidence has sometimes led people to believe Rumsey got the idea from Franklin. Indeed, if Franklin had wanted to make such a claim it likely would have been accepted, but he did not, and became one of Rumsey's supporters.
John Fitch had demonstrated his steamboat in Philadelphia the previous August. Although there was yet no overall patent system in the confederated states, he had patents from some states that gave him exclusive rights to his and any other steamboat. Rumsey was very protective of his designs and, though he was even more plagued by money problems, he sent the boat machinery to Philadelphia in March 1788. He quickly followed, armed with affidavits from those who had seen his steamboat or been involved with its creation. There was a pamphlet war with John Fitch. Some Philadelphia businessmen attempted to make the men set up a joint effort; but after years of travails and poverty, Fitch was not in a mood to compromise. When he said he would apply for a patent in England for Rumsey's water-tube boiler, Rumsey and others formed the Rumseian Society. They decided he should go to England to secure patents for his inventions and seek further financial backing.
After moving to England in 1788, Rumsey was able to take out four patents before his death there in 1792. While some of these relate to steamboats (like his water-tube boiler design, which made the steam engine much smaller and more efficient) most are concerned with hydrostatics and water power. His 1791 patent has all the pumps, motors, and hydraulic cylinders of fluid power engineering. By September 1792 he had a true water turbine, almost 40 years before it next appeared in France.
He spent four years in England. On December 20, 1792, on the eve of the demonstration of his new steamboat the Columbia Maid, he had just finished delivering a lecture to the Society of Mechanic Arts, when he was suddenly stricken with a severe pain in his head, and died the next morning. At the time his death was attributed to overstraining his brain. He was buried in London at Saint Margaret's Church.
In 1906 a second Rumseyan Society was formed in Shepherdstown. Through its efforts a monument to Rumsey was constructed in a park overlooking the Potomac.
Another Rumseian Society was formed in Shepherdstown in the 1980s to construct a replica of the successful Rumsey steamboat and celebrate the boat's bicentennial in 1987. The boat was constructed in the machine and blacksmith shop in the back of O'Hurley's General Store. The replica is housed in a small building behind the Entler Hotel. For a time, there was an annual regatta in Shepherdstown in early October in honor of Rumsey.
The bridge across the Potomac to Maryland is named after Rumsey, as is the James Rumsey Technical Institute in Hedgesville, West Virginia.