Kalpana Kalpana (Editor)

Total variation distance of probability measures

Updated on
Edit
Like
Comment
Share on FacebookTweet on TwitterShare on LinkedInShare on Reddit

In probability theory, the total variation distance is a distance measure for probability distributions. It is an example of a statistical distance metric, and is sometimes just called "the" statistical distance.

Contents

Definition

The total variation distance between two probability measures P and Q on a sigma-algebra F of subsets of the sample space Ω is defined via

δ ( P , Q ) = sup A ∈ F | P ( A ) − Q ( A ) | .

Informally, this is the largest possible difference between the probabilities that the two probability distributions can assign to the same event.

Special cases

For a finite or countable alphabet we can relate the total variation distance to the 1-norm of the difference of the two probability distributions as follows:

δ ( P , Q ) = 1 2 ∥ P − Q ∥ 1 = 1 2 ∑ x | P ( x ) − Q ( x ) | .

Similarly, for arbitrary sample space Ω , measure μ , and probability measures P and Q with Radon-Nikodym derivatives f P and f Q with respect to μ , an equivalent definition of the total variation distance is

δ ( P , Q ) = 1 2 ∥ f P − f Q ∥ L 1 ( μ ) = 1 2 ∫ Ω | f P − f Q | d μ .

Relationship with other concepts

The total variation distance is related to the Kullback–Leibler divergence by Pinsker's inequality.

References

Total variation distance of probability measures Wikipedia
(Text) CC BY-SA