Brunet was born in Belfort, then in the Alsace region of eastern France. His father was a military veterinary doctor. He graduated from the École Polytechnique in 1859, and joined the artillery school, graduating as a lieutenant in 1861.
Brunet participated in the French intervention in Mexico from August 1862 to June 1864, and received the Légion d'honneur in October 1864. In 1863 he was posted to the prestigious Horse Artillery Regiment of the Imperial Guard.
Napoleon III sent a group of military advisors to Japan to help modernize the Shogun's army. Brunet was sent as an artillery instructor, selected in September 1866. The mission arrived in early 1867 and trained the Shogun's troops for about a year. While in Japan, he was promoted to captain (August 1867). Then in 1868 the Shogun was overthrown in the Boshin War, and Emperor Meiji was nominally restored to full power. The French military mission was then ordered to leave Japan by Imperial decree.
However, Brunet chose to remain. He did not join his new posting in the French army, while not formally resigning, and left for the north of Japan with the remains of the Shogunate's armies in the hope of staging a counter-attack. In a letter to Napoleon III, Brunet explained the plan of the alliance, as well as his role in it:
"A revolution is forcing the Military Mission to return to France. Alone I stay, alone I wish to continue, under new conditions: the results obtained by the Mission, together with the Party of the North, which is the party favorable to France in Japan. Soon a reaction will take place, and the Daimyos of the North have offered me to be its soul. I have accepted, because with the help of one thousand Japanese officers and non-commissioned officers, our students, I can direct the 50,000 men of the Confederation."
Brunet took a very active role in the Boshin War. Brunet and Captain André Cazeneuve were present at the Battle of Toba-Fushimi (27–31 January 1868), near Osaka. After that Imperial victory, Brunet, Cazeneuve, and the Shogun's Admiral, Enomoto Takeaki, fled to Edo (now Tokyo) on the warship Fujisan.
When Edo also fell to the Imperial forces, Enomoto and Brunet fled to the northern island of Hokkaidō, where they proclaimed the Ezo Republic, with Enomoto as President. Brunet helped organize the Ezo army, under hybrid Franco-Japanese leadership. Otori Keisuke was Commander-in-chief, and Brunet was second in command. Each of the four brigades were commanded by a French officer (Fortant, Marlin, Cazeneuve, and Bouffier), with Japanese officers commanding each half-brigade. The final stand of the Shogun/Ezo forces was the Battle of Hakodate. The Ezo forces, numbering 3,000, were defeated by 7,000 Imperial troops.
In an interesting postscript to his involvement in the Boshin War, Brunet spoke highly of Shinsengumi vice-commander Hijikata Toshizō in his memoirs. Praising Hijikata's ability as a leader, he said that if the man had been in Europe, he most certainly would have been a general.
Brunet and the other French advisers were wanted by the Imperial government. But they were evacuated from Hokkaidō by a French warship (the corvette Coëtlogon, commanded by Dupetit-Thouars) and then taken to Saigon by the Dupleix. Brunet then returned to France. The new Japanese government requested that Brunet be punished for his activities in the Boshin War. But his actions had won popular support in France, and the request was denied.
Instead, he was suspended for six months and rejoined the French army in February 1870 with only a slight loss in seniority. During the Franco-Prussian War of 1870-1871, he was taken prisoner at the Siege of Metz. After the war, he played a key role as a member of the Versailles Army in the suppression of the Paris Commune in 1871. He was made an officer of the Légion d'honneur in September 1871 and posted as aide de camp to the Minister of War.
Brunet's former ally, Admiral Enomoto, had joined the Imperial government and became Minister of the Imperial Japanese Navy. Through Enomoto's influence the Imperial government not only forgave Brunet's actions but awarded him medals in May 1881 and again in March 1885; among those the Order of the Rising Sun. The medals were presented at the Japanese Embassy in Paris.
Brunet led a brilliant career in the French Army. As a colonel, he commanded the 11th Artillery Regiment between 1887 and 1891. Promoted to Brigade General in December 1891, he commanded the 48th Infantry Brigade between 1891 and 1897, then the 19th Artillery Brigade. In 1898, Chanoine, his former senior officer in the Japan mission, was Minister of War and Brunet became his Chief of Staff ("Chef de l'état-major du ministre de la Guerre") with the rank of Divisional General.
Brunet was a talented painter who left numerous depictions of his travels in Mexico and Japan.