Fitzhugh and his wife, Ann Bolling Randolph Fitzhugh (1747–1805), built Chatham Manor on property across the Rappahannock River from Fredericksburg, Virginia, completing it in 1771 after 3 years of construction. It still stands today as the National Park Service Headquarters for the Fredericksburg and Spotsylvania National Military Park. The Fitzhughs lived a lavish life there that included experimental farming and horse racing. After the Revolutionary War, as the economy floundered, Fitzhugh sold Chatham Manor and 1,288 acres (5.2 km2) to Churchill Jones for $20,000.
About 1799, William Fitzhugh bought the house at 607 Oronoco St., Alexandria, Virginia, that has become known as "The Boyhood Home of Robert E. Lee." The house was built in 1795 by John Potts, Jr. After William Fitzhugh's death, it then passed to William Henry Fitzhugh, his only son, and was rented to the Lee family.
Fitzhugh had built another mansion, Ravensworth, in 1796, where North Springfield, Virginia, is now located. This was his country home, with the Alexandria one being his townhouse. Ravensworth stood until about 1925, when it burned under mysterious circumstances.
William Fitzhugh and George Washington visited one another frequently until Washington's death in 1799, with Washington mentioning Fitzhugh in his diary and the two serving together on the Pohick Church vestry. Fitzhugh was the last person that Washington visited outside of Mount Vernon before his death in 1799.
Fitzhugh served in the Virginia House of Burgesses between 1772 and 1775. When that assembly was dissolved by Virginia colonial governor John Murray, 4th Earl of Dunmore, Fitzhugh continued to serve King George County, Virginia in ad hoc conventions held in Williamsburg and Richmond. He was also a member of the Revolutionary Committee of Safety in 1774-75. During the American Revolutionary War he was a commissioner of two arms and munitions factories. He served as a member of the Virginia House of Delegates in 1776–1777. He was a Virginia state senator in 1780–1787).
Fitzhugh was chosen as an elector for the 1789 election from Westmoreland District. That District consisted of King George County, Lancaster County, Northumberland County, Richmond County, Stafford County and Westmoreland County, which cover the area between the Rappahannock and Potomac Rivers. All of the 10 electors who voted cast one of their two votes for George Washington. 5 of them cast their other vote for John Adams. 3 cast theirs for George Clinton. 1 cast his for John Hancock. 1 cast his for John Jay. In a letter written by James Madison to Thomas Jefferson on March 29, 1789, 'Mr W Fitzhugh of Chatham' is described as a Federalist
He was the great-grandson of English immigrant Colonel William Fitzhugh who came to Virginia in about 1671 and owned 54,000 acres (220 km²) when he died in 1701. William of Chatham inherited most of the land. As a child he suffered the loss of an eye when accidentally hit with a horse riding crop by one of his stepbrothers. His mother was Lucy Carter, daughter of Robert "King" Carter.
In 1804 Fitzhugh's daughter Mary Lee Fitzhugh was married in the parlor of the Alexandria townhouse to George Washington Parke Custis, grandson of Martha Dandridge Custis Washington and adopted grandson of George Washington. In 1831 their daughter, Mary Anna Randolph Custis, married Robert E. Lee.
His wife was Ann Bolling Randolph (William Randolph II and Elizabeth Beverley's granddaughter). This Randolph connection made Ann Fitzhugh cousin to Thomas Jefferson, a visitor to their home in Fredericksburg.
Fitzhugh died in 1809 at the age of 69, leaving behind the three of his children who survived to adulthood. He was initially buried at Ravensworth, but when the mansion was destroyed, his remains and gravestone were moved to Lorton's Pohick Church cemetery.