Grimm, son of a paymaster, graduated from high school at Aschersleben in 1925 and then studied at the University of Halle, where he was instructed in prehistory, history, classical archaeology, German studies, geography and geology until 1929. He first participated in an excavation in 19 27 under Hans Hahne. From 1929 he was a research assistant at the State Bureau for Prehistory. In 1929 he was awarded a doctorate for his dissertation on Die vor- und frühgeschichtliche Besiedlung des Unterharzes und seines Vorlandes auf Grund der Bodenfunde (The Prehistoric and Protohistoric Settlement of Lower Harz and its Foothills on the basis of Archaeological Finds). His examiners were Hahne and Georg Karo.
Grimm was a member of the German Youth Movement and from 1926 to the völkisch youth league Adler und Falke (Eagle and Falcon). Well before the Nazi seizure of power he was a member of the Mannus-Society for "Aryan prehistory" and the prehistoric division of the Militant League for German Culture. On 1 February 1933, Grimm joined the Nazi Party and was registered as party member 1,447,316. He was a Blockleiter from 1933 to 1934. In 1935, Grimm was curator and acting director of the State Bureau for Volk Studies in Halle. From 1935 he was editor of the journal Mitteldeutsche Volkheit – Hefte für Vorgeschichte, Rassenkunde und Volkskunde (Central German Volk Studies – Volumes on Prehistory, Racial Studies and Volk Studies) along with the leader of the Volk Studies department, Heinz-Julius Niehoff. After he achieved his habilitation with his work on the Salzmünde Culture he was employed as Reader and Director of the State Bureau of Volk Studies in Halle from 1939 to 1945.
On 22 October 1940, Paul Grimm was called up for miulitary service. From January 1942 until November 1942 Grimm was an employee of the "Sonderstabs Vor- und Frühgeschichte" (Task force for Prehistory and Protohistory) at the Reichsleiter Rosenberg Taskforce. The Rosenberg Taskforce was a subdivision of the Nazi party tasked with confiscating artistic and cultural treasures throughout German-occupied Europe and to use them to advance Nazi ideology. Grimm was also leader of the State Agency for Prehistory and Protohistory in Kiev was first tasked with taking control of the various prehistoric artefacts in the Ukraine, especially those which could be used to argue for the role of Germanic peoples in Ukrainian prehistory, and with putting on an exhibition in Kiev for the Wehrmacht based on Nazi historical ideas. He organised the transfer of the Collection of the Kiev Institute of Archaeology from the destroyed Lavra to a new museum. In the following years, the Ukrainian and German sides praised his great personal service, as manifested in his sound scholarly work and his personal conduct towards Ukrainian museum staff.
From November 1942 until December 1944, Grimm was drafted into the army for a second time. After he was discharged due to sickness, Grimm was employed by the Institute for Eastern Studies in Schloss Höchstädt, Dillingen. There, with several colleagues, Grimm managed the most important depot of prehistoric and protohistoric artefacts and Volkish objects plundered from the occupied Soviet territories. Immediately before the arrival of the American army, Grimm was promoted to be Leader of the "Bergungsstätte Höchstädt" (Höchstädt Collection Point). Grimm surrendered the collections to the Americans and left Höchstädt for Halle in June 1945, without questioning.
Ernst Klee, Gunter Schöbel und Thomas Widera accuse Grimm of being at the very least an accomplice in the theft of artefacts from the occupied territories. Grimm had not demonstrably participated in the conveyance of Ukrainian artefacts to Höchstädt from Kiev, so this accusation is controversial. Joachim Herrmann argues that Grimm did not support the appropriation and even subverted it. Given Grimm's activities in Kiev and Höchstädt, however, it is impossible to deny his knowledge.
In December 1945, Grimm was dismissed from the public service for his membership in the Nazi party. On 26 February 1946, Paul Grimm was arrested by the NKVD and interned in the Special camp at Torgau. In December 1946 he was taken from there to Buchenwald, from which he was released on 3 February 1950. He was never sentenced by the Soviet side and was later invited to archaeological conferences in Russia and the Ukraine on several occasions. In 1951 he became a member of the East Berlin Academy of Sciences and worked in the Commission for Prehistory and Protohistory. After holding several teaching positions, he became Professor of the Humboldt University of Berlin in 1955 and from 1957 he was the assistant to the director of the Institute for Prehistory and Protohistory in the Academy of Sciences. In 1955 he was also made a member of the German Archaeological Institute.
From 1956 until 1972 he was editor in chief of the journal Ausgrabungen und Funde, Nachrichtenblatt für Vor- und Frühgeschichte (Excavations and Finds, Current Events in Prehistory and Protohistory). He was a member of the editorial board of Natur und Heimat (Nature and Home) from 1953 to 1962 and Zeitschrift für Archäologie (Journal of Archaeology) from its first volume in 1967 until 1973. Paul Grimm participated extensively in popular scholarship through articles, broschures and tours of excavations as well as through his membership in the Cultural Association of the DDR, for which he delivered numerous lectures about many subjects, especially the excavations at Tilleda.
Especially in his pioneering excavations of the abandoned Medieval village of Hohenrode and Tilleda Palace, the first entirely excavated Kaiserpfalz, Grimm was responsible for great advances in German archaeology. These excavations and especially his theoretical discussions made Grimm a pioneer of Medieval archaeology. He stressed that archaeological and literary sources had essentially equal authority, but nevertheless saw a close connection between the archaeological and historical data.Hohenrode, eine mittelalterliche Siedlung im Südharz. [Hohenrode, a Medieval Settlement in South Harz] Halle 1939 (= Veröffentlichungen der Landesanstalt für Volkheitskunde Halle, 11)
Die vor- und frühgeschichtlichen Burgwälle der Bezirke Halle und Magdeburg. [The Prehistoric and Protohistoric Earthworks in the Districts of Halle and Magdeberg] Akademie-Verlag, Berlin 1958 (= Handbuch vor- und frühgeschichtlicher Wall- und Wehranlagen, ed. by Wilhelm Unverzagt, Part 1 = Deutsche Akademie der Wissenschaften zu Berlin. Schriften der Sektion für Vor- und Frühgeschichte, Vol. 6)
Der Beitrag der Archäologie für die Erforschung des Mittelalters. [The Value of Archaeology for Research on the Middle Ages] (in Heinz A. Knorr (Ed.): Probleme des frühen Mittelalters in archäologischer und historischer Sicht, Akademie-Verlag, Berlin 1966, pp. 39–74)
Tilleda; eine Königspfalz am Kyffhäuser Teil 1. Die Hauptburg. [Tilleda; a Royal Palace at Kyffhäuser. Part 1. The Donjon] Akademie-Verlag, Berlin 1968 (= Deutsche Akademie der Wissenschaften zu Berlin. Schriften der Sektion für Vor- und Frühgeschichte, Vol. 24)
Tilleda; eine Königspfalz am Kyffhäuser Teil 2. Die Vorburg und Zusammenfassung. [Tilleda; a Royal Palace at Kyffhäuser. Part 2. The Bailey and Synopsis] Akademie-Verlag, Berlin 1990 (= Deutsche Akademie der Wissenschaften zu Berlin. Schriften der Sektion für Vor- und Frühgeschichte, Vol. 40)