Ambrose Wolfinger works as a "memory expert" for a manufacturing company's president; he keeps files of details about all the people President Malloy (Oscar Apfel) meets with, so that Malloy will never be embarrassed about not remembering things when meeting with them. Ambrose supports himself, his shrewish wife Leona (Kathleen Howard), his loving daughter Hope (from a previous marriage; played by Mary Brian), his freeloading brother-in-law Claude (Grady Sutton), and his abusive mother-in-law Cordelia (Vera Lewis).
At the start of the film, two burglars, played by Tammany Young and Walter Brennan, break into Ambrose's cellar late at night, get drunk on his homemade applejack, and start singing "On the Banks of the Wabash, Far Away"; Ambrose is forced to handle the situation, and he winds up being arrested for distilling liquor without a license. This is done on the order of the night court judge hearing the case. He forgets about dealing with the burglars. While on the way to the night court Ambrose talks about the wrestling match scheduled for that day and demonstrates an "unbreakable" hold on the neighborhood watch policeman who arrested the burglars. The policeman throws him into the street. When he asks Ambrose if he hurt him, Ambrose asks him how someone could be hurt by being dropped on his head.
The next day, Ambrose falsely tells Malloy that Cordelia had died from drinking poisoned liquor, and asks for the afternoon off to attend the funeral; in fact, he wants to go to see the big wrestling match. Malloy, touched by Ambrose's tale, lets him go for the day, and Ambrose's immediate supervisor, Mr. Peabody (Lucien Littlefield), tells all the other employees the tragic news so they can pay their respects to the family. In fact, Ambrose does not explicitly say that his mother-in-law died from poisoned liquor. Rather, when his employer asks him how she died, he begins to improvise a story. He says she was taken with a "chill" and that he poured her a drink. Then Malloy interrupts him and, assuming it was the liquor that killed her, says excitedly that he has read in the paper recently of many instances of people dying from poisoned liquor. Ambrose is too timid to contradict him.
Throughout that day, Ambrose has one problem after another: He has encounters with ticket-writing policemen and cars that are parked too close to his; he finds himself chasing a tire along railroad tracks and narrowly avoids getting hit by an electric interurban car; and while trying to get into the wrestling arena (Claude had stolen his ticket earlier), he gets knocked down by a wrestler who is thrown out of the building by his opponent.
Later that day, Ambrose comes home to find that Cordelia and Leona are furious about seeing Cordelia's obituary in the newspaper and receiving a huge amount of flowers, sympathy cards, and funeral wreaths. Claude sees Ambrose sprawled on the sidewalk after he is knocked over by a wrestler. Ambrose's secretary, who had been in the wrestling match audience (she said her mother is a friend of the contender, a Turk named Hookalaka Meshobbab) is bent over him expressing her concern over his injury. When Claude returns home ahead of Ambrose, he falsely tell his mother and his sister that he saw Ambrose and the secretary "drunk in the gutter". Furthermore, Peabody calls to say that Ambrose is fired because of his deception. Ambrose, who has been meek and mild through the entire film, finally has had enough, and in a rare moment of overt violence for Fields' characters, knocks Claude unconscious, and frightens his wife and mother-in-law into hiding. The angry Ambrose wants to beat them also ("I'll knock 'em for a row of lib-labs"), but soon he and his daughter leave the house to go live elsewhere.
Later, Malloy demands that Peabody rehire Ambrose because no one else can figure out Wolfinger's filing system; Hope answers the telephone, and says (falsely) that Ambrose has a better offer from another company. After some bargaining, Ambrose is rehired with a huge raise in pay and four weeks' vacation. Meanwhile, Leona realizes that she still loves Ambrose, scolds Claude for his laziness, and stands up to her disagreeable mother.
The film ends with Ambrose taking the family for a ride in his new car. Hope and Leona ride inside the car with him, while Claude and Cordelia ride in the rumble seat during a heavy rain.W. C. Fields as Ambrose Wolfinger
Mary Brian as Hope Wolfinger
Kathleen Howard as Leona Wolfinger
Grady Sutton as Claude Neselrode
Vera Lewis as Mrs. Cordelia Neselrode
Lucien Littlefield as Mr. Peabody
Oscar Apfel as President Malloy
Lew Kelly as Adolph Berg
Tammany Young as 'Willie' the Weasel
Walter Brennan as 'Legs' Garnett
Arthur Aylesworth as Night Court Judge
Tor Johnson as Tosoff, the 'Mad Russian', a wrestler
Harry Ekezian as Hookalakah Meshobbab, a wrestler
Carlotta Monti as "Ambrose's Secretary"
And Patrick H. O'Malley, Jr., James Flavin, Eddy Chandler, Edward Gargan, James Burke, Sam Lufkin, Billy Bletcher, George B. French, Rosemary Theby
Writing for The Spectator, Graham Greene characterized the film as "a slow worthy comedy".