The time is 1780, and Horatio Prim (Lou Costello) is a master tinker. He travels to the Kings Point estate of Tom Danbury (Jess Barker) with a letter of commendation from Gen. George Washington. He plans to present this letter to Danbury, hoping it will persuade the wealthy man to let Horatio marry Nora O'Leary (Anne Gillis), Danbury's housemaid. Unfortunately, Horatio has a romantic rival in Cuthbert Greenway (Bud Abbott), Danbury's butler, who is very fond of Nora and intends to prevent Horatio from presenting his letter, which Nora has taken for safekeeping.
Nora happens to overhear Danbury discussing his part in Benedict Arnold's plot; Danbury captures her, and hides the commendation letter in a secret compartment of the mantel clock. Danbury's fiancée, Melody Allen (Marjorie Reynolds), witnesses the situation and sets off on horseback to warn Washington's army. She enlists Horatio's help, but the two of them are mistakenly shot by American troops who are arriving at the estate. Their bodies are thrown down a well, and the soldiers ransack the house and burn it to the ground. The souls of the two unfortunates are condemned to remain on the estate until the "crack of doom" unless evidence of their innocence can be proved to the world.
For the next 166 years the ghosts of Horatio and Melody roam the grounds of the estate. Then, in the 1940s, the estate is restored by Sheldon Gage (John Shelton). When the restoration is finished, complete with the "original" furniture (which was removed before the estate's fateful burning), Sheldon invites some friends to spend the night there. Accompanying him are his psychiatrist, Dr. Ralph Greenway (Bud Abbott), a descendant of Cuthbert, as well as Sheldon's fiancée, June Prescott (Lynn Baggett) and her Aunt Millie (Binnie Barnes).
Upon arriving they are greeted by Emily (Gale Sondergaard), the maid who strongly believes that the estate is haunted. Ghosts Horatio and Melody have some fun with this idea and try to scare the guests (playing the harpsichord, turning on the radio full volume), especially Greenway whom Horatio mistakes for Cuthbert and hits with a candlestick. The newcomers hold a séance (during which Dr. Greenway is struck by Horatio for asking if he and Melody are "the two traitors" buried in the well) and learn the identities of the two ghosts, and of the letter which can free them (the spirit of Tom, channeling through Emily, relays the secret combination to open the clock and reveal the letter). They search for the letter but soon learn that not all of the furniture is original, as the clock that holds the letter sits in a New York museum. Greenway, as a way of atoning for the cruelty of his predecessor, travels to the museum to retrieve the letter. However, unexpected events force him to steal it. He arrives back at the estate with the state police in pursuit. Horatio uses the curse to his and Melody's advantage by riding in the police car that is supposed to take Greenway to jail; the car is thus prevented from leaving the estate, until the clock is opened.
Finally the letter is found, and Melody and Horatio leave the estate to enter heaven, each called by a loved one (Melody by Tom & Horatio by Nora). Unfortunately for Horatio, who is met at heaven's gate by Nora, he must wait one more day, as Nora points to a sign that says "Closed for Washington's Birthday".
The Time of Their Lives was filmed from March 6 through May 15, 1946.
As in the duo's preceding film, Little Giant, Abbott and Costello do not play friends or any kind of partners, but are simply individual members of the cast. Also as in the previous movie, Costello's character is largely the hero, while Abbott plays a somewhat unsympathetic dual role. The two comics' trademark vaudeville routines are absent, and in fact they speak directly to each other only during one scene at the beginning of the film. (This loosening of the comics' onscreen partnership was reportedly due to personal tensions that briefly broke up the act in 1945.) The film is also noted for having a somewhat darker and more serious tone than most films featuring the duo.
Abbott learned to drive a car for this film, which according to his son Bud Abbott, Jr., was the only time in his life that he ever drove.
A few weeks into filming, Costello wanted to switch roles with Abbott. He refused to work, but director Charles Barton waited him out. Costello eventually returned to work and said nothing more about it.
The film was re-released in 1951, along with Little Giant.
The film has been released three times on VHS: 1989, 1991, and 2000. It has also been released twice on DVD: The Best of Abbott and Costello Volume Two on May 4, 2004; and on October 28, 2008 as part of Abbott and Costello: The Complete Universal Pictures Collection.