Sasha (Igor Fomchenko) is a boy who lives with his mother (Marina Adzhubei) and his sister in an old house in Moscow. He is learning to play the violin. Every morning he has to cross the yard to go to the music school, trying to avoid some other children who are bullying and harassing him. This day he is lucky as Sergey (Vladimir Zamansky), the operator of a steamroller, tells them to leave Sasha alone.
At the music school he plays beautifully, but his teacher, who is more interested in form and order, is stifling his creativity with a metronome. On his way back home Sasha meets Sergey again, who allows him to help him on the steamroller. The two have lunch together and face a number of adventures as they walk around Moscow. They watch a wrecking ball demolishing a decrepit building, revealing one of the Seven Sisters in the background. Sergey tells stories about the war, and Sasha play the violin for his new friend.
They part with the plan to see a film together, but the plans are foiled by Sasha's mother. Sasha attempts to sneak out of the apartment, and in the final scene we see Sasha running after the steamroller in a dream like sequence.
The script for The Steamroller and the Violin was written by Andrei Tarkovsky and Andrei Konchalovsky, a fellow student of Tarkovsky at the State Institute of Cinematography (VGIK). They worked on the script for more than six months in 1959 and 1960. According to an interview with Polish journalist Zdzislaw Ornatowski, Tarkovsky wanted the film to be a poetic film, based on mood and atmosphere, not on sharp conflicts or traditional dramaturgy.
Apart from a couple of minor scenes, two scenes from the first script versions were not included in the final shooting script. First, a scene showing a rehearsal for the May Day parade showing tanks rolling along the street. In the final shooting script this scene is left out in favor of the demolition scene. This scene was intended to reference the war experience of Sergey. The second change involved the final scene of the film. In an earlier variant Sasha waits for Sergey to arrive at work. Sergey ignores Sasha, until Sasha climbs on the steamroller. An alarm clock rings, revealing this to be a dream.
For the role of the operator of the steamroller, Sergey, Tarkovsky cast Vladimir Zamansky, an actor at the Sovremennik Theatre. For the role of Sasha he cast Igor Fomchenko, a seven-year-old music school student. Although Tarkovsky was at this time only an unknown film student, he wanted to hire the well-known Sergey Urusevsky as cinematographer. As Urusevsky turned down the offer, Tarkovsky hired Vadim Yusov, who would also be the cameraman for Tarkovsky's Ivan's Childhood and Andrei Rublev.
Although the film was Tarkovsky's diploma film, it was produced at the Mosfilm studio at the production unit for children's films. The film was completed in 1960. For his diploma film, Tarkovsky earned the grade of excellent (Russian: отличный), the highest possible distinction.