When the O'Glendy Wrecking Company is contracted to demolish an old hotel, "cannonballer" Duke Mason shows up at the worksite, having been hired by boss Mike O'Glendy. Wrecker Tom Kemp accuses Duke of jinxing every project on which he works, and after foreman Matt Carney breaks up the ensuing fistfight, he assigns Duke, an arrogant risk-taker, to work with him on the building's high watertower. While aloft, Matt slips and desperately clings to the edge of the tower, and Duke casually jokes with him before pulling him to safety. Later, Mike, who took over the business after her husband's death, tells Matt that she has put up a bond to save the company, and that they only have four weeks to bring down the twelve-story building.
Wrecker Joe Poska then hosts a christening party after his wife Martha gives birth to a baby boy, but the wreckers have to leave suddenly to reinforce a section of unstable wall on the top of the hotel. The work goes smoothly until Joe accidentally falls to his death down the refuse chute while helping Duke dump a load of bricks. Joe's death reinforces Kemp's theory about the jinx, and Duke blames himself. While walking disconsolately on the street, Duke donates money to a missionary, who predicts that he will lighten someone's burden. Soon after, Duke rescues Peggy Starr when she is thrown out of a café for trying to steal money for food, then attempts to jump into the river. Duke introduces Peggy to his friends as his cousin, and attributes his new good luck to her. After Matt falls for Peggy, both he and Duke convince Mike to hire her as a secretary.
While work on the building continues, Peggy dates both Duke and Matt. Duke is promoted to the position of night foreman so that they can speed up demolition by maintaining two shifts. Kemp quits to avoid working with Duke, and while there are many near misses, no one is injured in an accident, convincing Duke that Peggy has broken his jinx. One day after breaking through a wall, Duke finds a cache of hidden money. Duke shares the money with his helper and his pal, Freddy Bunce, who needs money for arm surgery which would help him return to his true vocation as a baseball pitcher. Duke also generously gives Martha some money, saying that it is a donation from the night crew. Later, Matt proposes to Peggy, but she is hesitant to respond because she is torn between him and Duke.
One night, Matt catches Duke and Freddy showing off for an audience of women on the sidewalk, and warns Duke to put away the wrecking ball for which Duke is nicknamed. Duke secures the ball, but when he later refuses to rehire Kemp because he is drunk, Kemp angrily releases it. The runaway "cannonball" severely injures a worker, Emil, and causes structural damage to the building, resulting in the crew's having to reinforce the building before they can continue to take it down floor by floor. After Duke is blamed for the accident and threatens to quit, he asks Peggy to marry him. Convinced that Duke is acting irresponsibly, Peggy rejects him, prompting Duke to pick a fight with Matt, during which Mike learns that Peggy is not really his cousin. Mike breaks up the fight as a section of the hotel wall collapses, rendering the building unstable again. Duke leaves angrily while Matt and the crew try to stabilize the structure. As Matt climbs onto an unsteady girder, a hospital worker informs Mike that after regaining consciousness, Emil named Kemp as being responsible for his injury. Mike tries to call Duke, but he has already moved out of his boardinghouse. Duke again encounters the missionary, who advises him never to falter in helping a friend. Inspired by the advice, Duke returns to the worksite just as a supporting wall collapses and Matt is trapped under a girder. Duke climbs up to save Matt, but is then trapped with his friend when another section of the wall collapses. They are finally saved by Freddy, who, although recovering from his operation, risks his baseball career to throw them a lifeline. Some time later, Matt and Peggy begin their honeymoon at the Empire State Building, and Duke nearly falls off while he is kidding around with Mike.