A timeline of events related to information theory, quantum information theory and statistical physics, data compression, error correcting codes and related subjects.
1872 – Ludwig Boltzmann presents his H-theorem, and with it the formula Σpi log pi for the entropy of a single gas particle.1878 – J. Willard Gibbs defines the Gibbs entropy: the probabilities in the entropy formula are now taken as probabilities of the state of the whole system.1924 – Harry Nyquist discusses quantifying "intelligence" and the speed at which it can be transmitted by a communication system.1927 – John von Neumann defines the von Neumann entropy, extending the Gibbs entropy to quantum mechanics.1928 – Ralph Hartley introduces Hartley information as the logarithm of the number of possible messages, with information being communicated when the receiver can distinguish one sequence of symbols from any other (regardless of any associated meaning).1929 – Leó Szilárd analyses Maxwell's Demon, showing how a Szilard engine can sometimes transform information into the extraction of useful work.1940 – Alan Turing introduces the deciban as a measure of information inferred about the German Enigma machine cypher settings by the Banburismus process.1944 – Claude Shannon's theory of information is substantially complete.1947 – Richard W. Hamming invents Hamming codes for error detection and correction. For patent reasons, the result is not published until 1950.1948 – Claude E. Shannon publishes A Mathematical Theory of Communication1949 – Claude E. Shannon publishes Communication in the Presence of Noise – Nyquist–Shannon sampling theorem and Shannon–Hartley law1949 – Claude E. Shannon's Communication Theory of Secrecy Systems is declassified1949 – Robert M. Fano publishes Transmission of Information. M.I.T. Press, Cambridge, Mass. – Shannon–Fano coding1949 – Leon G. Kraft discovers Kraft's inequality, which shows the limits of prefix codes1949 – Marcel J. E. Golay introduces Golay codes for forward error correction1951 – Solomon Kullback and Richard Leibler introduce the Kullback–Leibler divergence1951 – David A. Huffman invents Huffman encoding, a method of finding optimal prefix codes for lossless data compression1953 – August Albert Sardinas and George W. Patterson devise the Sardinas–Patterson algorithm, a procedure to decide whether a given variable-length code is uniquely decodable1954 – Irving S. Reed and David E. Muller propose Reed–Muller codes1955 – Peter Elias introduces convolutional codes1957 – Eugene Prange first discusses cyclic codes1959 – Alexis Hocquenghem, and independently the next year Raj Chandra Bose and Dwijendra Kumar Ray-Chaudhuri, discover BCH codes1960 – Irving S. Reed and Gustave Solomon propose Reed–Solomon codes1962 – Robert G. Gallager proposes low-density parity-check codes; they are unused for 30 years due to technical limitations.1965 – Dave Forney discusses concatenated codes.1967 – Andrew Viterbi reveals the Viterbi algorithm, making decoding of convolutional codes practicable.1968 – Elwyn Berlekamp invents the Berlekamp–Massey algorithm; its application to decoding BCH and Reed–Solomon codes is pointed out by James L. Massey the following year.1968 – Chris Wallace and David M. Boulton publish the first of many papers on Minimum Message Length (MML) statistical and inductive inference1970 – Valerii Denisovich Goppa introduces Goppa codes1972 – J. Justesen proposes Justesen codes, an improvement of Reed–Solomon codes1973 – David Slepian and Jack Wolf discover and prove the Slepian–Wolf coding limits for distributed source coding.1976 – Gottfried Ungerboeck gives the first paper on trellis modulation; a more detailed exposition in 1982 leads to a raising of analogue modem POTS speeds from 9.6 kbit/s to 33.6 kbit/s.1976 – R. Pasco and Jorma J. Rissanen develop effective arithmetic coding techniques,1977 – Abraham Lempel and Jacob Ziv develop Lempel–Ziv compression (LZ77)1989 – Phil Katz publishes the .zip format including DEFLATE (LZ77 + Huffman coding); later to become the most widely used archive container and most widely used lossless compression algorithm1993 – Claude Berrou, Alain Glavieux and Punya Thitimajshima introduce Turbo codes1994 – Michael Burrows and David Wheeler publish the Burrows–Wheeler transform, later to find use in bzip21995 – Benjamin Schumacher coins the term qubit and proves the quantum noiseless coding theorem2001 – Sam Kwong and Yu Fan Ho proposed Statistical Lempel Ziv2006 – Jarek Duda introduced first entropy coder from Asymmetric Numeral Systems family, which later variants are replacing Huffman and arithmetic coding since 20142008 – Erdal Arıkan introduced polar codes, the first practical construction of codes that achieves capacity for a wide array of channels.