Born Giovanni Narcchione in Cleveland, Nardi began his mob work as an enforcer for the local vending machine workers union. He is the cousin of Anthony Delsanter, brother of Nicholas Nardi and father of John Nardi Jr. and Carol Nardi. Nardi would become a representative of his uncle Anthony Milano, a retired consigliere from the Cleveland crime family. He earned his first police record entry in 1939 at the age of twenty three. He had been employed by vending workers union to sell the services of their repair technicians. Sometimes he was too enthusiastic. When Nardi threatened a bar owner with bodily harm, the then Safety Director Eliot Ness ordered him to be arrested. Eventually, the charges were dropped.
Nardi soon became business partners with Ohio Teamsters official William Presser, a mob associate and father of future Teamsters president Jackie Presser in several Jukebox companies.
By the 1940s, Nardi had become a member of the Vending Machine Service Employees Local 410, part of the Teamsters Union. He soon became secretary-treasurer of the Local. Nardi also formed ties with Aladino "Jimmy the Weasel" Fratianno a future boss with the Los Angeles crime family with whom he also ran a bookmaking operation in Cleveland's Little Italy.
Nardi soon built numerous street rackets such as drug trafficking, extortion, labor racketeering, arms trafficking, illegal gambling, and loan sharking.
Nardi could have enjoyed a bright future with the Cleveland family, but he was too independent and ambitious to accept its structure. Not content to wait years to become a made man, or full member, of the organization, Nardi eventually stopped paying tribute to the family. By the late 1960s, the Cleveland family was losing patience with Nardi due to his independence and his ties with Danny Greene, the boss of Cleveland's Irish mob. In 1976, Nardi returned from Florida where he successfully defended himself against federal narcotics and gun-running charges. His uncle, Anthony Milano, was hoping to have his son, Peter, return from the West Coast to work with Nardi. Upon his return, he approached Danny Greene for an alliance. The Cleveland crime family had already made several attempts on Nardi's life and Nardi needed to find allies. Greene saw that Nardi's street rackets would be a valuable addition to his organization.
In 1976, the smouldering dispute between the Cleveland family and the Greene-Nardi alliance broke into open warfare. Both sides started planting car bombs in mobsters' cars. The warfare escalated with the murder of Cleveland consigliere Leo Moceri. Each year the Cleveland family ran the Feast of the Assumption festival in the Little Italy section of Cleveland. At the end of the 1976 festival, Nardi claimed that the Cleveland family owed him a share of the illegal gambling profits from that event. Moceri publicly denied Nardi's claim and the two sides exchanged threats. In the summer of 1976, Moceri disappeared; in August his Mercedes-Benz sports car was found soaked in blood. Greene and Nardi then went after Eugene "the Animal" Ciasullo, the family's most feared enforcer. Ciasullo was seriously injured by a bomb placed on his front porch.
In 1976, after the Moceri murder, Licavoli and new underboss Angelo Lonardo went to New York to talk to Anthony Salerno, the titular head of the New York Genovese crime family. The two Cleveland mobsters wanted Salerno's help in murdering Greene and Nardi. Nardi and Greene had previously taken a trip to New York to discuss a partnership with Gambino crime family boss Paul Castellano about a meat business venture in Texas. Salerno agreed to speak to Castellano and to have Nardi and Greene murdered on their next trip to New York. However, neither Greene or Nardi travelled to New York again.
There were two murder attempts on Nardi's life by Cleveland family mafiosi Butchie Cisternino and Allie Calabrese prior to his eventual murder. They tried to assassinate Nardi in Little Italy with a high-powered rifle. Another attempt was made a few days later when a shotgun blast was fired at Nardi from a moving car. In response to these murder attempts, Nardi threatened that everyone responsible for taking shots at him would be killed.
Just weeks before his death, Nardi granted an interview to a reporter inquiring about a rumor that Licavoli and he were feuding. During the interview, Nardi stated that he and Licavoli were lifelong friends and vehemently denied the allegations that there was a feud between them. He also denied that Danny Greene worked for him stating that they were just friends.
On May 17, 1977, in Cleveland, a bomb was placed in a car next to Nardi's vehicle in the rear of the parking lot of the Teamsters Joint Council 41, across from the musicians union. When Nardi left his office and entered into his vehicle, the bomb was detonated by remote control. The impact from the explosion had blown away both of Nardi's legs. According to the book To Kill the Irishman by Rick Porrello, as Nardi was being pulled away from the wreckage, Nardi whispered "It didn't hurt" in a final act of defiance. He was pronounced dead within minutes.
In the 2011 film Kill the Irishman, John Nardi (Vincent D'Onofrio) is depicted as a caporegime rather than as an associate of the family. The Five Families' decision to appoint James Licavoli as boss of the family and the latter's attempts to confiscate Nardi's rackets leads the Nardi crew to fully align themselves with Danny Greene. Soon after Genovese boss Anthony Salerno (Paul Sorvino) sends Ray Ferritto from Los Angeles to Cleveland, Nardi is fatally injured by a car bomb moments after telling Greene that they will soon "take this town over." A heartbroken Greene is then shown cradling his dying friend. In reality, Danny Greene was not present when Nardi was murdered.