In mathematics, more specifically measure theory, there are various notions of the convergence of measures. For an intuitive general sense of what is meant by convergence of measures, consider a sequence of measures μn on a space, sharing a common collection of measurable sets. Such a sequence might represent an attempt to construct 'better and better' approximations to a desired measure μ that is difficult to obtain directly. The meaning of 'better and better' is subject to all the usual caveats for taking limits; for any error tolerance ε > 0 we require there be N sufficiently large for nN to ensure the 'difference' between μn and μ is smaller than ε. Various notions of convergence specify precisely what the word 'difference' should mean in that description; these notions are not equivalent to one another, and vary in strength.

Three of the most common notions of convergence are described below.

Informal descriptions

edit

This section attempts to provide a rough intuitive description of three notions of convergence, using terminology developed in calculus courses; this section is necessarily imprecise as well as inexact, and the reader should refer to the formal clarifications in subsequent sections. In particular, the descriptions here do not address the possibility that the measure of some sets could be infinite, or that the underlying space could exhibit pathological behavior, and additional technical assumptions are needed for some of the statements. The statements in this section are however all correct if μn is a sequence of probability measures on a Polish space.

The various notions of convergence formalize the assertion that the 'average value' of each 'sufficiently nice' function should converge:  

To formalize this requires a careful specification of the set of functions under consideration and how uniform the convergence should be.

The notion of weak convergence requires this convergence to take place for every continuous bounded function f. This notion treats convergence for different functions f independently of one another, i.e., different functions f may require different values of Nn to be approximated equally well (thus, convergence is non-uniform in f).

The notion of setwise convergence formalizes the assertion that the measure of each measurable set should converge:  

Again, no uniformity over the set A is required. Intuitively, considering integrals of 'nice' functions, this notion provides more uniformity than weak convergence. As a matter of fact, when considering sequences of measures with uniformly bounded variation on a Polish space, setwise convergence implies the convergence   for any bounded measurable function f[citation needed]. As before, this convergence is non-uniform in f.

The notion of total variation convergence formalizes the assertion that the measure of all measurable sets should converge uniformly, i.e. for every ε > 0 there exists N such that   for every n > N and for every measurable set A. As before, this implies convergence of integrals against bounded measurable functions, but this time convergence is uniform over all functions bounded by any fixed constant.

Total variation convergence of measures

edit

This is the strongest notion of convergence shown on this page and is defined as follows. Let   be a measurable space. The total variation distance between two (positive) measures μ and ν is then given by

 

Here the supremum is taken over f ranging over the set of all measurable functions from X to [−1, 1]. This is in contrast, for example, to the Wasserstein metric, where the definition is of the same form, but the supremum is taken over f ranging over the set of measurable functions from X to [−1, 1] which have Lipschitz constant at most 1; and also in contrast to the Radon metric, where the supremum is taken over f ranging over the set of continuous functions from X to [−1, 1]. In the case where X is a Polish space, the total variation metric coincides with the Radon metric.

If μ and ν are both probability measures, then the total variation distance is also given by

 

The equivalence between these two definitions can be seen as a particular case of the Monge–Kantorovich duality. From the two definitions above, it is clear that the total variation distance between probability measures is always between 0 and 2.

To illustrate the meaning of the total variation distance, consider the following thought experiment. Assume that we are given two probability measures μ and ν, as well as a random variable X. We know that X has law either μ or ν but we do not know which one of the two. Assume that these two measures have prior probabilities 0.5 each of being the true law of X. Assume now that we are given one single sample distributed according to the law of X and that we are then asked to guess which one of the two distributions describes that law. The quantity

 

then provides a sharp upper bound on the prior probability that our guess will be correct.

Given the above definition of total variation distance, a sequence μn of measures defined on the same measure space is said to converge to a measure μ in total variation distance if for every ε > 0, there exists an N such that for all n > N, one has that[1]

 

Setwise convergence of measures

edit

For   a measurable space, a sequence μn is said to converge setwise to a limit μ if

 

for every set  .

Typical arrow notations are   and  .

For example, as a consequence of the Riemann–Lebesgue lemma, the sequence μn of measures on the interval [−1, 1] given by μn(dx) = (1 + sin(nx))dx converges setwise to Lebesgue measure, but it does not converge in total variation.

In a measure theoretical or probabilistic context setwise convergence is often referred to as strong convergence (as opposed to weak convergence). This can lead to some ambiguity because in functional analysis, strong convergence usually refers to convergence with respect to a norm.

Weak convergence of measures

edit

In mathematics and statistics, weak convergence is one of many types of convergence relating to the convergence of measures. It depends on a topology on the underlying space and thus is not a purely measure-theoretic notion.

There are several equivalent definitions of weak convergence of a sequence of measures, some of which are (apparently) more general than others. The equivalence of these conditions is sometimes known as the Portmanteau theorem.[2]

Definition. Let   be a metric space with its Borel  -algebra  . A bounded sequence of positive probability measures   on   is said to converge weakly to a probability measure   (denoted  ) if any of the following equivalent conditions is true (here   denotes expectation or the   norm with respect to  , while   denotes expectation or the   norm with respect to  ):

  •   for all bounded, continuous functions  ;
  •   for all bounded and Lipschitz functions  ;
  •   for every upper semi-continuous function   bounded from above;
  •   for every lower semi-continuous function   bounded from below;
  •   for all closed sets   of space  ;
  •   for all open sets   of space  ;
  •   for all continuity sets   of measure  .

In the case   with its usual topology, if   and   denote the cumulative distribution functions of the measures   and  , respectively, then   converges weakly to   if and only if   for all points   at which   is continuous.

For example, the sequence where   is the Dirac measure located at   converges weakly to the Dirac measure located at 0 (if we view these as measures on   with the usual topology), but it does not converge setwise. This is intuitively clear: we only know that   is "close" to   because of the topology of  .

This definition of weak convergence can be extended for   any metrizable topological space. It also defines a weak topology on  , the set of all probability measures defined on  . The weak topology is generated by the following basis of open sets:

 

where

 

If   is also separable, then   is metrizable and separable, for example by the Lévy–Prokhorov metric. If   is also compact or Polish, so is  .

If   is separable, it naturally embeds into   as the (closed) set of Dirac measures, and its convex hull is dense.

There are many "arrow notations" for this kind of convergence: the most frequently used are  ,  ,   and  .

Weak convergence of random variables

edit

Let   be a probability space and X be a metric space. If Xn: Ω → X is a sequence of random variables then Xn is said to converge weakly (or in distribution or in law) to the random variable X: Ω → X as n → ∞ if the sequence of pushforward measures (Xn)(P) converges weakly to X(P) in the sense of weak convergence of measures on X, as defined above.

Comparison with vague convergence

edit

Let   be a metric space (for example   or  ). The following spaces of test functions are commonly used in the convergence of probability measures.[3]

  •   the class of continuous functions   each vanishing outside a compact set.
  •   the class of continuous functions   such that  
  •   the class of continuous bounded functions

We have  . Moreover,   is the closure of   with respect to uniform convergence.[3]

Vague Convergence

edit

A sequence of measures   converges vaguely to a measure   if for all  ,  .

Weak Convergence

edit

A sequence of measures   converges weakly to a measure   if for all  ,  .

In general, these two convergence notions are not equivalent.

In a probability setting, vague convergence and weak convergence of probability measures are equivalent assuming tightness. That is, a tight sequence of probability measures   converges vaguely to a probability measure   if and only if   converges weakly to  .

The weak limit of a sequence of probability measures, provided it exists, is a probability measure. In general, if tightness is not assumed, a sequence of probability (or sub-probability) measures may not necessarily converge vaguely to a true probability measure, but rather to a sub-probability measure (a measure such that  ).[3] Thus, a sequence of probability measures   such that   where   is not specified to be a probability measure is not guaranteed to imply weak convergence.

Weak convergence of measures as an example of weak-* convergence

edit

Despite having the same name as weak convergence in the context of functional analysis, weak convergence of measures is actually an example of weak-* convergence. The definitions of weak and weak-* convergences used in functional analysis are as follows:

Let   be a topological vector space or Banach space.

  1. A sequence   in   converges weakly to   if   as   for all  . One writes   as  .
  2. A sequence of  converges in the weak-* topology to   provided that   for all  . That is, convergence occurs in the point-wise sense. In this case, one writes   as  .

To illustrate how weak convergence of measures is an example of weak-* convergence, we give an example in terms of vague convergence (see above). Let   be a locally compact Hausdorff space. By the Riesz-Representation theorem, the space   of Radon measures is isomorphic to a subspace of the space of continuous linear functionals on  . Therefore, for each Radon measure  , there is a linear functional   such that   for all  . Applying the definition of weak-* convergence in terms of linear functionals, the characterization of vague convergence of measures is obtained. For compact  ,  , so in this case weak convergence of measures is a special case of weak-* convergence.

See also

edit

Notes and references

edit
  1. ^ Madras, Neil; Sezer, Deniz (25 Feb 2011). "Quantitative bounds for Markov chain convergence: Wasserstein and total variation distances". Bernoulli. 16 (3): 882–908. arXiv:1102.5245. doi:10.3150/09-BEJ238. S2CID 88518773.
  2. ^ Klenke, Achim (2006). Probability Theory. Springer-Verlag. ISBN 978-1-84800-047-6.
  3. ^ a b c Chung, Kai Lai (1974). A course in probability theory. Internet Archive. New York, Academic Press. pp. 84–99. ISBN 978-0-12-174151-8.

Further reading

edit