Fisk was born in the hamlet of Pownal, Vermont, in Bennington County in 1835. After a brief period in school, he ran away in 1850 and joined Van Amberg's Mammoth Circus & Menagerie. Later, he became a hotel waiter, and finally adopted the business of his father, a peddler. He applied what he learned in the circus to his peddling and grew his father's business. He then became a salesman for Jordan Marsh, a Boston dry goods firm.
A failure as a salesman, he was sent to Washington, D.C., in 1861 to sell textiles to the government. By his shrewd dealing in army contracts during the Civil War, and, by some accounts, cotton smuggling across enemy lines – in which he enlisted the help of his father – he accumulated considerable wealth, which he soon lost in speculation.
In 1864 Fisk became a stockbroker in New York City, and was employed by Daniel Drew as a buyer. He aided Drew in the Erie War against Cornelius Vanderbilt for control of the Erie Railroad. This resulted in Fisk and Jay Gould becoming members of the Erie directorate, and subsequently, a well-planned raid netted Fisk and Gould control of the railroad. The association with Gould continued until Fisk's death.
Fisk and Gould carried financial buccaneering to extremes: their program included an open alliance with New York politician Boss Tweed, the wholesale bribery of legislatures, and the buying of judges. Their attempt to corner the gold market culminated in the fateful Black Friday of September 24, 1869. Though many investors were ruined, Fisk and Gould escaped significant financial harm.
Fisk married Lucy Moore in 1854, when he was 19 and she was 15. Lucy was an orphan, raised by an uncle from Springfield, Massachusetts. She tolerated Fisk's many extramarital affairs, perhaps because she was happy living with her own love, Fanny Harrod, in Boston. Regardless, they remained close, with Fisk visiting her every few weeks and spending summers and vacations with her every chance he could.
In New York, Fisk had a relationship with Josie Mansfield. Mansfield was considered a voluptuous beauty by Victorian standards of female desirability. Fisk housed Mansfield in an apartment a few doors down from the Erie Railroad headquarters on West 23rd Street and had a covered passage built linking the back doors of the headquarters and her apartment building. Fisk's relationship with Mansfield scandalized New York society.
Mansfield eventually fell in love with Fisk's business associate Edward Stiles Stokes (1840–1901), a man noted for his good looks. Stokes left his wife and family, and Mansfield left Fisk.
In a bid for money, Mansfield and Stokes tried to extort Fisk by threatening the publication of letters written by Fisk to Mansfield that allegedly proved Fisk's legal wrongdoings. A legal and public relations battle followed, but Fisk refused to pay Mansfield anything. Increasingly frustrated and flirting with bankruptcy, Stokes confronted Fisk in New York City on January 6, 1872, in the Grand Central Hotel and shot him twice, in the arm and abdomen. A relatively young man of 37, Fisk died of the abdominal wound the next morning after giving a dying declaration identifying Stokes as the killer.
Stokes pleaded self-defense, using a wildly incongruent set of mitigating circumstances. He claimed to have been suffering from emotional turmoil at the time he committed the act. Fisk's death was blamed on medical malpractice by those who treated his mortal wound. Stokes was subsequently tried three times for the Fisk murder. The first trial where he was charged with first degree murder ended in a hung jury, and rumors of jury members bribed. The second trial found him guilty of first degree murder and he was sentenced to death, a verdict overturned by appeal. The third trial concluded with a conviction for manslaughter, and Stokes served four years of a six-year prison sentence in Sing Sing Penitentiary.
Fisk's body was laid out for public view in the Grand Opera House, which he had owned. Some twenty thousand people came to pay their respects with five times as many more individuals waiting in the streets to gain entrance. The notorious letters Fisk had written to Mansfield, numbering thirty-nine, were published in the New York Herald one week after his death. For those who had hoped to read revelations detailing Fisk's corrupt business practices, the letters were a sorry disappointment, being no more than the commonplace communications between a man and the woman he loved. Fisk is buried in the Prospect Hill Cemetery in Brattleboro, Vermont.
Fisk was vilified by high society for his amoral and eccentric ways and by many pundits of the day for his business dealings; but he was loved and mourned by the workingmen of New York and the Erie Railroad. He was known as "Colonel" for being the nominal commander of the 9th New York National Guard Infantry Regiment, although his only experience of military action with this unit was an inglorious role in the Orange Riot of July 12, 1871.
Fisk's life was fictionalized in the biopic The Toast of New York (1937), starring Edward Arnold as Fisk. He may be the inspiration for the character "Big Jim" in Bob Dylan's song "Lily, Rosemary, and The Jack of Hearts". In Phoebe Atwood Taylor's Figure Away, a woman says that "Jim Fisk was shot with [a revolver] just like" the one that she owns.
The Colt House Revolver is known among collectors as the Jim Fisk model or the Jim Fisk pistol, as it was used by Edward Stiles Stokes (Ned Stokes) in his murder.
The circumstances surrounding his murder were dramatized in the CBS radio program Crime Classics on June 29, 1953 in the episode entitled "The Checkered Life and Sudden Death of Colonel James Fisk".