Alexandre César Victor Charles Destutt de Tracy was born in Paris on 9 September 1781. His father was Antoine Destutt de Tracy, an aristocrat, philosopher and deputy in the Estates-General for Moulins, Allier. Victor Destutt de Tracy entered the École Polytechnique in 1791, and graduated as an engineer. He was appointed lieutenant in 1800. In 1804 he was appointed second captain in the fourth sapper battalion. Destutt de Tracy served at Boulogne, with the French Army of Italy in Dalmatia, and in 1807 in the Engineering headquarters.
Destutt de Tracy was appointed battalion commander in the 58th infantry in 1807. He was made aide-de-camp to General Sebastiani, ambassador to Constantinople, and followed him to Spain. He fought with distinction at the Battle of Almonacid (1809) and Battle of Albuera (1811), where he was wounded. He served in the French invasion of Russia of 1812, where he was taken prisoner and interned in Saint Petersburg. In 1814 Destutt de Tracy was freed in an exchange and promoted to colonel. In 1816 he married the widow of General Louis-Michel Letort de Lorville. He left the army in 1818 to pursue scientific studies. He was formally granted retirement as Staff Colonel of Infantry on 5 April 1820.
On 5 August 1822 Destutt de Tracy was elected as deputy for the 2nd electoral district of Allier (Montluçon) in a by-election. He sat on the far left next to Lafayette, whose son Georges Washington de La Fayette had married his sister Emilie de Tracy, and voted with the opposition. He ran for re-election on 25 February 1824 but did not succeed. On 17 November 1827 he was elected deputy for Moulins, Allier, and took his seat with the opposition.
Destutt de Tracy was re-elected on 23 June 1830 and supported the establishment of the July Monarchy, but always maintained his independence of the royal party. He was made an Officer of the Legion of Honour in 1831. He was re-elected as deputy in 1831, 1834, 1838, 1839, 1842 and 1846. As a member of the opposition during the reign of Louis Philippe he was moderate but held strong moral opinions. He demanded abolition of the death penalty, abolition of the hereditary peerage, abolition of the slave trade and freedom of slaves, free education and the abandonment of Algeria. In 1841 he was appointed a member of the Board of Agriculture.
In the French Revolution of 1848 Destutt de Tracy was a colonel of the 1st legion of the National Guard of Paris. On 23 April 1848 he was elected to represent Orne in the Constituent Assembly, where he was a member of the Finance committee. Surprisingly, he voted for the banishment of the Orleans royal family. He voted for the abolition of the death penalty, but afterwards voted with the right. During the June Days Uprising (23–26 June 1848) he acted against the insurgents at the head of the 1st legion of the National Guard.
When Prince Louis Napoleon was elected president, Destutt de Tracy was made Minister of the Navy and Colonies from 20 December 1848 to 2 June 1849. While he was in office slavery was abolished in the colonies. The death penalty for political crimes was also abolished. On 13 May 1849 Destutt de Tracy was elected representative for Orne in the Legislative Assembly. He continued as minister in the second cabinet of Odilon Barrot from 2 June 1849 to 31 October 1849, when he left office. In the Assembly he continued to vote with the monarchist majority, and was opposed to Napoleon's ambitions. He protested against the coup d'état of 2 December 1851. In the elections of 29 February 1852 he ran for election to the Assembly as an independent candidate for the 2nd district of Orne, and was decisively defeated.
Victor Destutt de Tracy left politics in 1852 and retired to Paray-le-Frésil, Allier, where he took up agriculture. He died there on 13 March 1864, aged 82.