Well-behaved statistic

(Redirected from Well behaving statistics)

Although the term well-behaved statistic often seems to be used in the scientific literature in somewhat the same way as is well-behaved in mathematics (that is, to mean "non-pathological"[1][2]) it can also be assigned precise mathematical meaning, and in more than one way. In the former case, the meaning of this term will vary from context to context. In the latter case, the mathematical conditions can be used to derive classes of combinations of distributions with statistics which are well-behaved in each sense.

First Definition: The variance of a well-behaved statistical estimator is finite and one condition on its mean is that it is differentiable in the parameter being estimated.[3]

Second Definition: The statistic is monotonic, well-defined, and locally sufficient.[4]

Conditions for a Well-Behaved Statistic: First Definition

edit

More formally the conditions can be expressed in this way.   is a statistic for   that is a function of the sample,  . For   to be well-behaved we require:

 : Condition 1

  differentiable in  , and the derivative satisfies:

 : Condition 2

Conditions for a Well-Behaved Statistic: Second Definition

edit

In order to derive the distribution law of the parameter T, compatible with  , the statistic must obey some technical properties. Namely, a statistic s is said to be well-behaved if it satisfies the following three statements:

  1. monotonicity. A uniformly monotone relation exists between s and ? for any fixed seed   – so as to have a unique solution of (1);
  2. well-defined. On each observed s the statistic is well defined for every value of ?, i.e. any sample specification   such that   has a probability density different from 0 – so as to avoid considering a non-surjective mapping from   to  , i.e. associating via   to a sample   a ? that could not generate the sample itself;
  3. local sufficiency.   constitutes a true T sample for the observed s, so that the same probability distribution can be attributed to each sampled value. Now,   is a solution of (1) with the seed  . Since the seeds are equally distributed, the sole caveat comes from their independence or, conversely from their dependence on ? itself. This check can be restricted to seeds involved by s, i.e. this drawback can be avoided by requiring that the distribution of   is independent of ?. An easy way to check this property is by mapping seed specifications into  s specifications. The mapping of course depends on ?, but the distribution of   will not depend on ?, if the above seed independence holds – a condition that looks like a local sufficiency of the statistic S.

The remainder of the present article is mainly concerned with the context of data mining procedures applied to statistical inference and, in particular, to the group of computationally intensive procedure that have been called algorithmic inference.

Algorithmic inference

edit

In algorithmic inference, the property of a statistic that is of most relevance is the pivoting step which allows to transference of probability-considerations from the sample distribution to the distribution of the parameters representing the population distribution in such a way that the conclusion of this statistical inference step is compatible with the sample actually observed.

By default, capital letters (such as U, X) will denote random variables and small letters (u, x) their corresponding realizations and with gothic letters (such as  ) the domain where the variable takes specifications. Facing a sample  , given a sampling mechanism  , with   scalar, for the random variable X, we have

 

The sampling mechanism  , of the statistic s, as a function ? of   with specifications in   , has an explaining function defined by the master equation:

 

for suitable seeds   and parameter ?

Example

edit

For instance, for both the Bernoulli distribution with parameter p and the exponential distribution with parameter ? the statistic   is well-behaved. The satisfaction of the above three properties is straightforward when looking at both explaining functions:   if  , 0 otherwise in the case of the Bernoulli random variable, and   for the Exponential random variable, giving rise to statistics

 

and

 

Vice versa, in the case of X following a continuous uniform distribution on   the same statistics do not meet the second requirement. For instance, the observed sample   gives  . But the explaining function of this X is  . Hence a master equation   would produce with a U sample   and a solution  . This conflicts with the observed sample since the first observed value should result greater than the right extreme of the X range. The statistic   is well-behaved in this case.

Analogously, for a random variable X following the Pareto distribution with parameters K and A (see Pareto example for more detail of this case),

 

and

 

can be used as joint statistics for these parameters.

As a general statement that holds under weak conditions, sufficient statistics are well-behaved with respect to the related parameters. The table below gives sufficient / Well-behaved statistics for the parameters of some of the most commonly used probability distributions.

Common distribution laws together with related sufficient and well-behaved statistics.
Distribution Definition of density function Sufficient/Well-behaved statistic
Uniform discrete    
Bernoulli    
Binomial    
Geometric    
Poisson    
Uniform continuous    
Negative exponential    
Pareto    
Gaussian    
Gamma    

References

edit
  1. ^ Dawn Iacobucci. "Mediation analysis and categorical variables: The final frontier" (PDF). Retrieved 7 February 2017.
  2. ^ John DiNardo; Jason Winfree. "The Law of Genius and Home Runs Refuted" (PDF). Retrieved 7 February 2017.
  3. ^ A DasGupta. "(no title)" (PDF). Retrieved 7 February 2017. {{cite web}}: Cite uses generic title (help)
  4. ^ Apolloni, B; Bassis, S.; Malchiodi, D.; Witold, P. (2008). The Puzzle of Granular Computing. Studies in Computational Intelligence. Vol. 138. Berlin: Springer.