Smith was born in Troy, Alabama and raised in Birmingham, Alabama. He received his nickname as a child from his liking for climbing trees. In 1920 he moved to Pittsburgh, Pennsylvania, where he worked as an entertainer before touring on the T. O. B. A. vaudeville circuit, performing as a singer and comedian as well as a pianist. For a time he worked as accompanist for blues singer Ma Rainey and Butterbeans and Susie.
In the mid-1920s he was recommended by Cow Cow Davenport to J. Mayo Williams at Vocalion Records, and in 1928 he moved, with his wife and young son, to Chicago, Illinois to record. For a time he, Albert Ammons, and Meade Lux Lewis lived in the same rooming house.
On 29 December 1928 he recorded his influential "Pine Top's Boogie Woogie", one of the first "boogie woogie" style recordings to make a hit, and which cemented the name for the style. Pine Top talks over the recording, telling how to dance to the number. He said he originated the number at a house-rent party in St. Louis, Missouri. Smith was the first ever to direct "the girl with the red dress on" to "not move a peg" until told to "shake that thing" and "mess around". Similar lyrics are heard in many later songs, including "Mess Around" and "What'd I Say" by Ray Charles.
Smith was scheduled to make another recording session for Vocalion in 1929, but died from a gunshot wound in a dance-hall fight in Chicago the day before the session. Sources differ as to whether he was the intended recipient of the bullet. "I saw Pinetop spit blood" was the famous headline in Down Beat magazine.
At least two photographs of Smith are known to exist.
Smith was acknowledged by other boogie-woogie pianists such as Albert Ammons and Pete Johnson as a key influence, and he gained posthumous fame when "Boogie Woogie" was arranged for big band and recorded by Tommy Dorsey and His Orchestra in 1938. Although not immediately successful, "Boogie Woogie" was so popular during and after World War II that it became Dorsey's best-selling record, with over five million copies sold. Bing Crosby (recorded January 21, 1946 with Lionel Hampton's Orchestra) and Count Basie also issued their versions of the song.
From the 1950s, Joe Willie Perkins became universally known as "Pinetop Perkins" for his recording of "Pinetop's Boogie Woogie". Perkins later became Muddy Waters's pianist. When he was in his nineties, he recorded a song on his 2004 album Ladies' Man, which played on the by-then common misconception that he had written "Pinetop's Boogie Woogie".
Ray Charles adapted "Pine Top's Boogie Woogie" for his song "Mess Around", for which the authorship was credited to "A. Nugetre", Ahmet Ertegun.
In 1975, the Bob Thiele Orchestra recorded a modern jazz album called I Saw Pinetop Spit Blood, which included a treatment of "Pinetop's Boogie Woogie" as well as the title song.
Gene Taylor recorded a version of "Pinetop's Boogie Woogie" on his eponymous 2003 album.
Claes Oldenburg, the pop artist, proposed a Pinetop Smith Monument in his book Proposals for Monuments and Buildings 1965–69. Oldenburg described the monument as "a wire extending the length of North Avenue, west from Clark Street, along which at intervals runs an electric impulse colored blue so that there’s one blue line as far as the eye can see. Pinetop Smith invented boogie woogie blues at the corner of North and Larrabee, where he finally was murdered: the electric wire is 'blue' and dangerous."
Smith was a posthumous 1991 inductee of the Alabama Jazz Hall of Fame.