Budd was born on May 18, 1851 in Janesville, Wisconsin, to Joseph H. and Lucinda (Ash) Budd. While in Wisconsin, Joseph was a lawyer and ran Western Novelty Works, a company which put out farm machinery. When Budd was seven years old, he and his family emigrated to the West, settling in Stockton, California in 1858. Budd attended local grammar and high schools before attending the University of California, Berkeley, where he was admitted to the Zeta Psi fraternity, graduating in 1873. Following his graduation, Budd returned to Stockton to study law, being admitted to the California Bar the following year. His public career in the courts began after being appointed the San Joaquin County deputy district attorney.
During the 1884 elections, the Democratic Party nominated Budd as its candidate for the 2nd California district in the U.S. House of Representatives. His campaign for the House was based on fighting railroad corruption, going as far as to refuse any travel on railroads altogether during the campaign, earning Budd the nickname "Buckboard Jim." Budd won the election, becoming the first Democrat to win the district since the Civil War.
Once in the U.S. Congress, Budd was influential in gaining enough congressional support for appropriating federal money to fund the dredging of the Stockton Channel, assisting shipping into the Port of Stockton along the San Joaquin River. Budd served a single term in the U.S. House of Representatives, refusing the Democratic Party nomination for the 1886 elections.
For the 1894 state general elections, Budd was nominated by the state Democratic convention as the party's nominee for the governorship. During the campaign, Budd targeted the growing influence of the Southern Pacific Railroad and rail monopolies in general, proposing that the "only solution to the railroad question was that the [state] government own a line" with slight nationalization, as well as reducing railroad fares by 25%. Budd also advocated a property assessment ceiling cap of 45 cents per $100 in order to save state expenditures with the looming Long Depression.
During the often bitter campaign for the governorship, Budd faced serious allegations brought forth by Republican-sympathizing newspapers that he had raped a teenage maid, Nancy Neff, in 1876. The papers alleged that Neff had become pregnant, giving birth to a child that shortly died afterwards. After giving birth, Neff was alleged to have contracted smallpox and sent by Budd to a hospice to die, but not before Budd had forced her to relinquish all her property to him. Budd fully denied all the allegations, branding his betrayal of Neff as an "infamous falsehood" imagined by Republican party machines. Budd did not deny he knew Neff, yet claimed instead that he had always acted as a brother to Neff and in no way harmed her.
Despite the serious charges, Budd's campaign remained largely successful. He gained favorable attention during the campaign by personally quieting a runaway team of horses, as well as helping extinguish a fire in Willows.
In the final results, Budd edged out Republican Party challenger Morris M. Estee by 0.4%, or roughly 1,200 votes. The strong third place showing of Jonathan V. Webster of the People's Party at 18% helped tip the vote balance into Budd's favor. It was the first gubernatorial election where the Australian ballot was used in California.
Due in part to his narrow electoral victory, Republicans alleged Budd had engaged in voter fraud. The California State Assembly investigated these claims through a special committee. The Assembly committee's findings concluded that no evidence of vote wrongdoing could be found.
Budd became the 19th Governor of California on January 11, 1895. In the early stage of his governorship, Budd encouraged improvements to the Australian ballot, consolidating or eliminating superfluous state departments and positions to save revenue, and amend the Constitution of California in order to create stricter qualification requirements for State Railroad Commission members. Budd desired a Railroad Commission substantially less influenced by the Southern Pacific and other rail monopolies.
During his governorship, Budd pushed legislation through the California State Legislature in 1895 to create the Bureau of Highways, an agency to construct and manage maintenance to the state's growing road network. The Bureau of Highways would later evolve into the California Department of Transportation (Caltrans). The following year, Budd authorized the purchase of the Lake Tahoe Wagon Road, making it the first state highway.
Budd's push for future legislation, however, was slowed down by the Legislature, which was controlled by an increasingly hostile Republican majority. Although Budd was able to reduce state expenditures gradually, including closing down the State Printing Office, his reform program for increased railroad oversight was halted by the opposition, limiting his ability to make any long-term political reforms.
With his health failing, Budd decided not to seek re-election in 1898. Budd would be the last Democrat to occupy the Governor's Office until Culbert Olson in 1939.
Following the end of his first and only term, Budd relocated to San Francisco, opening a law practice in the city. He served as an attorney to the Board of State Harbor Commissioners. In 1900, Budd was appointed by Governor Henry Gage as a member of the Regents of the University of California.
Budd died at the age of 57 on July 30, 1908 in Stockton, and is buried in the city's Rural Cemetery.
The Budd Center and Budd Shops at San Joaquin Delta College in Stockton are named after the governor.