Total variation distance of probability measures

  (Redirected from Total variation distance)

In probability theory, the total variation distance is a distance measure for probability distributions. It is an example of a statistical distance metric, and is sometimes called the statistical distance or variational distance.

DefinitionEdit

The total variation distance between two probability measures P and Q on a sigma-algebra   of subsets of the sample space   is defined via[1]

 

Informally, this is the largest possible difference between the probabilities that the two probability distributions can assign to the same event.

PropertiesEdit

Relation to other distancesEdit

The total variation distance is related to the Kullback–Leibler divergence by Pinsker's inequality:

 

When the set is countable, the total variation distance is related to the L1 norm by the identity:[2]

 

Connection to transportation theoryEdit

The total variation distance (or half the norm) arises as the optimal transportation cost, when the cost function is  , that is,

 

where the expectation is taken with respect to the probability measure   on the space where   lives, and the infimum is taken over all such   with marginals   and  , respectively[3].

See alsoEdit

ReferencesEdit

  1. ^ Chatterjee, Sourav. "Distances between probability measures" (PDF). UC Berkeley. Archived from the original (PDF) on July 8, 2008. Retrieved 21 June 2013.
  2. ^ David A. Levin, Yuval Peres, Elizabeth L. Wilmer, 'Markov Chains and Mixing Times', 2nd. rev. ed. (AMS, 2017), Proposition 4.2, p. 48.
  3. ^ Villani, Cédric (2009). Optimal Transport, Old and New. Grundlehren der mathematischen Wissenschaften. 338. Springer-Verlag Berlin Heidelberg. p. 10. doi:10.1007/978-3-540-71050-9. ISBN 978-3-540-71049-3.