Connolly was born to poor Irish immigrants from the Aran Islands, fisherman John Connolly and Ann O'Donnell, as one of twelve children, in South Boston, Massachusetts. Growing up at a time when the parks and playground movement in Boston was slowly developing, Connolly joined other boys in the streets and vacant lots to run, jump, and play ball.
He was educated at Notre Dame Academy and then at the Mather and Lawrence grammar school, but never went to high school. Instead, Connolly worked as a clerk with an insurance company in Boston and later with the United States Army Corps of Engineers in Savannah, Georgia.
His predisposition to sport also became apparent. Calling a special meeting of the Catholic Library Association (CLA) of Savannah in 1891, he was instrumental in forming a football team. Soon thereafter, Connolly was elected captain of the CLA Cycling Club and aggressively sought to promote the sport on behalf of the Savannah Wheelmen.
Altogether dissatisfied with his career path, Connolly sought to regain the lost years of high school through self-education. In October 1895, he sat for the entrance examination to the Lawrence Scientific School and was unconditionally accepted to study the classics at Harvard University.
After the creation of the International Olympic Committee in 1894 the first modern edition of the Olympic Games were scheduled for April 6 to 15, 1896 in Athens, Greece. Connolly decided to participate, and submitted a request for a leave of absence to the Chairman of the Harvard University Committee on the Regulation of Athletic Sports and was denied. According to Connolly himself, he was informed that his only course of action would be to resign and make a reapplication to the College. Connolly then claimed to have replied:
I am not resigning and I'm not making application to re-enter. I'm getting through with Harvard right now. Good day!
It is unclear whether this really happened. Harvard records do show a request by Connolly for a leave of absence to Europe, which was denied. Connolly then requested an honorable withdrawal as a student, which was granted on March 19, 1896.
Representing the Suffolk Athletic Club, which paid for most of Connolly's expenses (Connolly later claimed he paid it all himself), he left for Greece on a German freighter, the Barbarossa, along with most of the rest of the first American Olympic team. After arriving in Naples, Italy he was robbed and almost lost his ticket to Athens. He managed to retrieve it only after a pursuit against the robber. Finally he took the train to Athens, arriving there just in time for the Games.
The first final on the opening day was the triple jump (then known as the hop, skip and jump), one of the events in which Connolly competed. Connolly's style, taking two hops with the right foot, is no longer allowed in this event nowadays, but was perfectly acceptable in 1896. With this style, he outjumped the field, finishing more than one meter ahead of his nearest opponent by jumping 13.71 m (44 ft 11 3/4 in), earning him the first silver medal (gold medals did not yet exist). With this performance, he became the first Olympic champion since AD 385, when the Athenian Zopyrus won the pankration (other sources name the Armenian Varasdates, who won at boxing in 369).
He went on to take second place in the high jump (1.65 m / 5 ft 5 in) tying with Robert Garrett behind Ellery Clark, and third place in the long jump (5.84 m / 19 ft 2 in). Back home in Boston, Connolly was welcomed enthusiastically, and was presented a gold watch by the citizens of South Boston.
Connolly would also visit the second edition of the modern Olympics, held in Paris. There, he failed to retain his title in the triple jump, losing to compatriot Meyer Prinstein.
In the 1984 NBC miniseries The First Olympics: Athens 1896, he was portrayed by David Caruso. Contrary to what is portrayed in the film, Connolly did not emigrate to America from Ireland.
Connollystrasse, in Munich is named in his honour and was a key location in the events surrounding the Munich Massacre at the 1972 Summer Olympics.
The 1904 Summer Olympics were also attended by Connolly, but as a journalist, not as an athlete. Earlier, he had already published his accounts of the Spanish–American War in the Boston Globe as Letters from the Front in Cuba. He served there in the Irish 9th Infantry of Massachusetts.
Connolly became an authority on maritime writing, after spending years on many different vessels, fishing boats, military ships all over the world. In all, he published more than 200 short stories, and 25 novels. Furthermore, he twice ran for Congress of the United States on the ticket of the Progressive Party, but never was elected.
He never returned to Harvard, but received an honorary athletic sweater in 1948. A year later, he was offered an honorary doctorate by Harvard University, which he turned down. Connolly died in New York City at the age of 88. A collection of items related to Connolly, including his triple jump silver medal, is housed in the library of Colby College in Maine.
Connolly was a crew member of the victorious schooner Esperanto in 1920, during the first International Fishing Schooner Championship Races in Halifax, Nova Scotia. He wrote of this in Collier's Weekly on December 25, 1920 and in The Book of the Gloucester Fishermen, published in 1927.