Data processing inequality - Alchetron, the free social encyclopedia

The Data processing inequality is an information theoretic concept which states that the information content of a signal cannot be increased via a local physical operation. This can be expressed concisely as 'post-processing cannot increase information'. As explained by Kinney and Atwal, the DPI means that information is generally lost (never gained) when transmitted through a noisy channel.

Example

Let be a Markov chain X → Y → Z
Then,
I ( x ; y ) ⩾ I ( x ; z ) with
I ( x ; y ) = I ( x ; z ) if and only if X → Z → Y
where I ( x ; y ) is the Mutual information

References

Data processing inequality Wikipedia

(Text) CC BY-SA