Foster was born in Greene County, Missouri.
He was a staunch Unionist whose brother Marshall was murdered by secessionists in early 1861 on his way to vote. During the American Civil War, on Emory formed a Unionist Home Guard company called "Foster's Mounted Rangers" in which he served as captain, enlisting on August 28, 1861. He later enlisted in the Federally funded Missouri State Militia, being elected major of the 7th MSM Cavalry. He and his men engaged in skirmishes around his new home in Warrensburg, Missouri and Foster gained a reputation as an aggressive commander.
On August 15, 1862 after a two-day march from Warrensburg, Missouri to Lexington, Missouri, he was ordered to take 800 men on a 20-mile march to Lone Jack, Missouri to engage Confederate troops that were attempting to capture Jackson County, Missouri in what would become the Battle of Lone Jack.
Upon arrival, Foster's force encountered an 800-1,600 man sleeping Confederate recruiting force under Colonel John T. Coffee and Lieutenant Colonel John Charles Tracy and routed them. However, the firing of Foster's artillery alerted other Confederate recruiting commands in the area of his presence and intent. Confederates under Colonels Vard Cockrell, Upton Hays, and DeWitt C. Hunter were joined by Lt. Col. Tracy and a fierce five-hour battle ensued the next morning. The Federals withdrew after Foster was wounded and Col. Coffee's command joined Cockrell.
Foster and his brother were severely wounded, unable to withdraw, and were taken to a cabin. The cabin was captured by the Confederates and Foster was about to be executed by a member of Quantrill's Raiders when an 18-year-old Cole Younger physically threw the gunman out sparing Foster and his brothers life. They gave $1,000 and their handguns to Younger who then delivered them to the Foster sons' mother in Warrensburg (all despite Younger's being a member of the Confederates).
In 1876, Younger as a member of the James-Younger Gang was captured in the botched Northfield, Minnesota bank robbery. Foster was to forcefully argue for a parole for Younger in the 1890s. Also arguing for the parole was future Secretary of War Stephen Benton Elkins whose only taste of combat had been at Lone Jack, an experience which he said filled him with disgust of war.
After the war he became an editor at the St. Louis Journal.
On September 4, 1875 he fought a duel north of Rockford, Illinois with Edwards, who was then an editor of the St. Louis Times (after leaving the Kansas City Times in 1873).
The dispute had centered on an August 25 article by Edwards talking about the mistreatment of Jefferson Davis at the Winnebago County, Illinois Fair. The Journal replied the same day "the writer of the Times article had lied, and knew he lied, when he wrote it."
Edwards demanded a retraction and Foster refused saying the editorial was not directed at Edwards personally. On August 30 Edwards challenged Foster to a duel:
The disclaimer in the first four paragraphs of your letter would be satisfactory had you followed it up by a withdrawal of the offensive terms of your editorial, so far as they referred to me personally. But as you decline to do so I must, therefore, construe your letter of this date, and its spirit, as a refusal on your part to do me an act of common justice, and so regarding it, I deem it my duty to ask of you that satisfaction which one gentleman has a right to ask of another.
Their seconds made the arrangements and at 5 pm the two met in a field and both missed as Foster smoked a cigar. Foster commented, "A little high."
Edwards demanded a second fire, "I will go on if it takes a thousand fires."
Foster refused a second fire. He had been challenged and shots had been fired and so his honor had been maintained.
Both shook hands and made a bourbon toast.
Foster died in Oakland, California.