Open main menu

Normally distributed and uncorrelated does not imply independent

In probability theory, although simple examples illustrate that linear uncorrelatedness of two random variables does not in general imply their independence, it is sometimes mistakenly thought that it does imply that when the two random variables are normally distributed. This article demonstrates that assumption of normal distributions does not have that consequence, although the multivariate normal distribution, including the bivariate normal distribution, does.

To say that the pair of random variables has a bivariate normal distribution means that every linear combination of and for constant (i.e. not random) coefficients and has a univariate normal distribution. In that case, if and are uncorrelated then they are independent.[1] However, it is possible for two random variables and to be so distributed jointly that each one alone is marginally normally distributed, and they are uncorrelated, but they are not independent; examples are given below.

Contents

ExamplesEdit

A symmetric exampleEdit

 
Joint range of   and  . Darker indicates higher value of the density function.

Suppose   has a normal distribution with expected value 0 and variance 1. Let   have the Rademacher distribution, so that   or  , each with probability 1/2, and assume   is independent of  . Let  . Then

  •   and   are uncorrelated;
  • both have the same normal distribution; and
  •   and   are not independent.[2]

To see that   and   are uncorrelated, one may consider the covariance  : by definition, it is

 

Then by definition of the random variables  ,  , and  , and the independence of   from  , one has

 

To see that   has the same normal distribution as  , consider

 

(since   and   both have the same normal distribution), where   is the cumulative distribution function of the normal distribution..

To see that   and   are not independent, observe that   or that  .

Finally, the distribution of the simple linear combination   concentrates positive probability at 0:  . Therefore, the random variable   is not normally distributed, and so also   and   are not jointly normally distributed (by the definition above).

An asymmetric exampleEdit

 
The joint density of   and  . Darker indicates a higher value of the density.

Suppose   has a normal distribution with expected value 0 and variance 1. Let

 

where   is a positive number to be specified below. If   is very small, then the correlation   is near   if   is very large, then   is near 1. Since the correlation is a continuous function of  , the intermediate value theorem implies there is some particular value of   that makes the correlation 0. That value is approximately 1.54. In that case,   and   are uncorrelated, but they are clearly not independent, since   completely determines  .

To see that   is normally distributed—indeed, that its distribution is the same as that of   —one may compute its cumulative distribution function:

 

where the next-to-last equality follows from the symmetry of the distribution of   and the symmetry of the condition that  .

In this example, the difference   is nowhere near being normally distributed, since it has a substantial probability (about 0.88) of it being equal to 0. By contrast, the normal distribution, being a continuous distribution, has no discrete part—that is, it does not concentrate more than zero probability at any single point. Consequently   and   are not jointly normally distributed, even though they are separately normally distributed.[3]

Examples with support almost everywhere in ℝ2Edit

It is well-known that the ratio   of two independent standard normal random deviates   and   has a Cauchy distribution. One can equally well start with the Cauchy random variable   and derive the conditional distribution of   to satisfy the requirement that   with   and   independent and standard normal. Cranking through the math one finds that

 

in which   is a Rademacher random variable and   is a Chi-squared random variable with two degrees of freedom.

Consider two sets of  ,  . Note that   is not indexed by   – that is, the same Cauchy random variable   is used in the definition of both   and  . This sharing of   results in dependences across indices: neither   nor   is independent of  . Nevertheless all of the   and   are uncorrelated as the bivariate distributions all have reflection symmetry across the axes.

 
Non-normal joint distributions with normal marginals.

The figure shows scatterplots of samples drawn from the above distribution. This furnishes two examples of bivariate distributions that are uncorrelated and have normal marginal distributions but are not independent. The left panel shows the joint distribution of   and  ; the distribution has support everywhere but at the origin. The right panel shows the joint distribution of   and  ; the distribution has support everywhere except along the axes and has a discontinuity at the origin: the density diverges when the origin is approached along any straight path except along the axes.

See alsoEdit

ReferencesEdit

  1. ^ Hogg, Robert; Tanis, Elliot (2001). "Chapter 5.4 The Bivariate Normal Distribution". Probability and Statistical Inference (6th ed.). pp. 258–259. ISBN 0130272949.
  2. ^ UIUC, Lecture 21. The Multivariate Normal Distribution, 21.6:"Individually Gaussian Versus Jointly Gaussian".
  3. ^ Edward L. Melnick and Aaron Tenenbein, "Misspecifications of the Normal Distribution", The American Statistician, volume 36, number 4 November 1982, pages 372–373