Open main menu

Wikipedia β

In statistics, the delta method is a result concerning the approximate probability distribution for a function of an asymptotically normal statistical estimator from knowledge of the limiting variance of that estimator. It was first described in 1938 by Robert Dorfman[1]

Contents

Univariate delta methodEdit

While the delta method generalizes easily to a multivariate setting, careful motivation of the technique is more easily demonstrated in univariate terms. Roughly, if there is a sequence of random variables Xn satisfying

 

where θ and σ2 are finite valued constants and   denotes convergence in distribution, then

 

for any function g satisfying the property that g′(θ) exists and is non-zero valued.

Proof in the univariate caseEdit

Demonstration of this result is fairly straightforward under the assumption that g′(θ) is continuous. To begin, we use the mean value theorem (i.e.: the first order approximation of a Taylor series using Taylor's theorem):

 

where   lies between Xn and θ. Note that since   and  , it must be that   and since g′(θ) is continuous, applying the continuous mapping theorem yields

 

where   denotes convergence in probability.

Rearranging the terms and multiplying by   gives

 

Since

 

by assumption, it follows immediately from appeal to Slutsky's theorem that

 

This concludes the proof.

Proof with an explicit order of approximationEdit

Alternatively, one can add one more step at the end, to obtain the order of approximation:

 

This suggests that the error in the approximation converges to 0 in probability.

Multivariate delta methodEdit

By definition, a consistent estimator B converges in probability to its true value β, and often a central limit theorem can be applied to obtain asymptotic normality:

 

where n is the number of observations and Σ is a (symmetric positive semi-definite) covariance matrix. Suppose we want to estimate the variance of a function h of the estimator B. Keeping only the first two terms of the Taylor series, and using vector notation for the gradient, we can estimate h(B) as

 

which implies the variance of h(B) is approximately

 

One can use the mean value theorem (for real-valued functions of many variables) to see that this does not rely on taking first order approximation.

The delta method therefore implies that

 

or in univariate terms,

 

Example: the binomial proportionEdit

Suppose Xn is Binomial with parameters   and n. Since

 

we can apply the Delta method with g(θ) = log(θ) to see

 

Hence, even though for any finite n, the variance of   does not actually exist (since Xn can be zero), the asymptotic variance of   does exist and is equal to

 

Note that since p>0,   as  , so with probability converging to one,   is finite for large n.

Moreover, if   and   are estimates of different group rates from independent samples of sizes n and m respectively, then the logarithm of the estimated relative risk   has asymptotic variance equal to

 

This is useful to construct a hypothesis test or to make a confidence interval for the relative risk.

NoteEdit

The delta method is often used in a form that is essentially identical to that above, but without the assumption that Xn or B is asymptotically normal. Often the only context is that the variance is "small". The results then just give approximations to the means and covariances of the transformed quantities. For example, the formulae presented in Klein (1953, p. 258) are:

 

where hr is the rth element of h(B) and Biis the ith element of B.

See alsoEdit

ReferencesEdit

External linksEdit