Hilikus44

Contributions I have made to Wikipedia

One-Dimensional Examples of Sufficient Statistics

Normal Distribution

If $X_{1},...,X_{n}$ are independent and normally distributed with expected value θ (a parameter) and known finite variance $\sigma ^{2}$, then $T(X_{1}^{n})={\overline {X}}={\frac {1}{n}}\sum _{i=1}^{n}X_{i}$ is a sufficient statistic for θ.

To see this, consider the joint probability density function of $X_{1}^{n}=(X_{1},...,X_{n})$. Because the observations are independent, the pdf can be written as a product of individual densities, ie -

${\begin{aligned}f_{X_{1}^{n}}(x_{1}^{n})&=\prod _{i=1}^{n}{\tfrac {1}{\sqrt {2\pi \sigma ^{2}}}}\,e^{\frac {-(x_{i}-\theta )^{2}}{2\sigma ^{2}}}=(2\pi \sigma ^{2})^{-n \over 2}\,e^{{-1 \over 2\sigma ^{2}}\sum _{i=1}^{n}(x_{i}-\theta )^{2}}=(2\pi \sigma ^{2})^{-n \over 2}\,e^{{-1 \over 2\sigma ^{2}}\sum _{i=1}^{n}((x_{i}-{\overline {x}})-(\theta -{\overline {x}}))^{2}}=(2\pi \sigma ^{2})^{-n \over 2}\,e^{{-1 \over 2\sigma ^{2}}(\sum _{i=1}^{n}(x_{i}-{\overline {x}})^{2}+\sum _{i=1}^{n}(\theta -{\overline {x}})^{2}-2\sum _{i=1}^{n}(x_{i}-{\overline {x}})(\theta -x_{i}))}.\end{aligned}}$

Then, since $\sum _{i=1}^{n}(x_{i}-{\overline {x}})(\theta -x_{i})=0$, which can be shown simply by expanding this term,

${\begin{aligned}f_{X_{1}^{n}}(x_{1}^{n})&=(2\pi \sigma ^{2})^{-n \over 2}\,e^{{-1 \over 2\sigma ^{2}}(\sum _{i=1}^{n}(x_{i}-{\overline {x}})^{2}+n(\theta -{\overline {x}})^{2})}&=(2\pi \sigma ^{2})^{-n \over 2}\,e^{{-1 \over 2\sigma ^{2}}\sum _{i=1}^{n}(x_{i}-{\overline {x}})^{2}}\,e^{{-n \over 2\sigma ^{2}}(\theta -{\overline {x}})^{2}}.\end{aligned}}$

The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting

${\begin{aligned}h(x_{1}^{n})=(2\pi \sigma ^{2})^{-n \over 2}\,e^{{-1 \over 2\sigma ^{2}}\sum _{i=1}^{n}(x_{i}-{\overline {x}})^{2}},\,\,\,g_{\theta }(x_{1}^{n})=e^{{-n \over 2\sigma ^{2}}(\theta -{\overline {x}})^{2}}.\end{aligned}}$

Since $h(x_{1}^{n})$ does not depend on the parameter $\theta$ and $g_{\theta }(x_{1}^{n})$ depends only on $x_{1}^{n}$ through the function $T(X_{1}^{n})={\overline {X}}={\frac {1}{n}}\sum _{i=1}^{n}X_{i},$

the Fisher–Neyman factorization theorem implies $T(X_{1}^{n})={\overline {X}}={\frac {1}{n}}\sum _{i=1}^{n}X_{i}$ is a sufficient statistic for $\theta$.

Exponential Distribution

If $X_{1},...,X_{n}$ are independent and exponentially distributed with expected value θ (an unknown real-valued positive parameter), then $T(X_{1}^{n})=\sum _{i=1}^{n}X_{i}$ is a sufficient statistic for θ.

To see this, consider the joint probability density function of $X_{1}^{n}=(X_{1},...,X_{n})$. Because the observations are independent, the pdf can be written as a product of individual densities, ie -

${\begin{aligned}f_{X_{1}^{n}}(x_{1}^{n})&=\prod _{i=1}^{n}{1 \over \theta }\,e^{{-1 \over \theta }x_{i}}={1 \over \theta ^{n}}\,e^{{-1 \over \theta }\sum _{i=1}^{n}x_{i}}.\end{aligned}}$

The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting

${\begin{aligned}h(x_{1}^{n})=1,\,\,\,g_{\theta }(x_{1}^{n})={1 \over \theta ^{n}}\,e^{{-1 \over \theta }\sum _{i=1}^{n}x_{i}}.\end{aligned}}$

Since $h(x_{1}^{n})$ does not depend on the parameter $\theta$ and $g_{\theta }(x_{1}^{n})$ depends only on $x_{1}^{n}$ through the function $T(X_{1}^{n})=\sum _{i=1}^{n}X_{i}$

the Fisher–Neyman factorization theorem implies $T(X_{1}^{n})=\sum _{i=1}^{n}X_{i}$ is a sufficient statistic for $\theta$.

Two-Dimensional Examples of Sufficient Statistics

Uniform Distribution (with two parameters)

If $X_{1},...,X_{n}\,$ are independent and uniformly distributed on the interval $[\alpha ,\beta ]\,$ (where $\alpha \,$ and $\beta \,$ are unknown parameters), then $T(X_{1}^{n})=(\min _{1\leq i\leq n}X_{i},\max _{1\leq i\leq n}X_{i})\,$ is a two-dimensional sufficient statistic for $(\alpha \,,\,\beta )$.

To see this, consider the joint probability density function of $X_{1}^{n}=(X_{1},...,X_{n})$. Because the observations are independent, the pdf can be written as a product of individual densities, ie -

${\begin{aligned}f_{X_{1}^{n}}(x_{1}^{n})&=\prod _{i=1}^{n}({1 \over \beta -\alpha })\mathbf {1} _{\{\alpha \leq x_{i}\leq \beta \}}&=({1 \over \beta -\alpha })^{n}\mathbf {1} _{\{\alpha \leq x_{i}\leq \beta ,\,\forall \,i=1,\cdots ,n\}}&=({1 \over \beta -\alpha })^{n}\mathbf {1} _{\{\alpha \,\leq \,\min _{1\leq i\leq n}X_{i}\}}\mathbf {1} _{\{\max _{1\leq i\leq n}X_{i}\,\leq \,\beta \}}.\end{aligned}}$

The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting

${\begin{aligned}h(x_{1}^{n})=1,\,\,\,g_{(\alpha \,,\,\beta )}(x_{1}^{n})=({1 \over \beta -\alpha })^{n}\mathbf {1} _{\{\alpha \,\leq \,\min _{1\leq i\leq n}X_{i}\}}\mathbf {1} _{\{\max _{1\leq i\leq n}X_{i}\,\leq \,\beta \}}.\end{aligned}}$

Since $h(x_{1}^{n})$ does not depend on the parameter $(\alpha \,,\,\beta )$ and $g_{(\alpha \,,\,\beta )}(x_{1}^{n})$ depends only on $x_{1}^{n}$ through the function $T(X_{1}^{n})=(\min _{1\leq i\leq n}X_{i},\max _{1\leq i\leq n}X_{i})\,$,

the Fisher–Neyman factorization theorem implies $T(X_{1}^{n})=(\min _{1\leq i\leq n}X_{i},\max _{1\leq i\leq n}X_{i})\,$ is a sufficient statistic for $(\alpha \,,\,\beta )$.

Gamma Distribution

If $X_{1},...,X_{n}\,$ are independent and distributed as a $\Gamma (\alpha \,,\,\beta )\,\,$ , where $\alpha \,$ and $\beta \,$ are unknown parameters of a Gamma distribution, then $T(X_{1}^{n})=(\prod _{i=1}^{n}{x_{i}},\sum _{i=1}^{n}{x_{i}})\,$ is a two-dimensional sufficient statistic for $(\alpha \,,\,\beta )$.

To see this, consider the joint probability density function of $X_{1}^{n}=(X_{1},...,X_{n})$. Because the observations are independent, the pdf can be written as a product of individual densities, ie -

${\begin{aligned}f_{X_{1}^{n}}(x_{1}^{n})&=\prod _{i=1}^{n}({1 \over \Gamma (\alpha )\beta ^{\alpha }})x_{i}^{\alpha -1}e^{{-1 \over \beta }x_{i}}&=({1 \over \Gamma (\alpha )\beta ^{\alpha }})^{n}(\prod _{i=1}^{n}x_{i})^{\alpha -1}e^{{-1 \over \beta }\sum _{i=1}^{n}{x_{i}}}.\end{aligned}}$

The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting

${\begin{aligned}h(x_{1}^{n})=1,\,\,\,g_{(\alpha \,,\,\beta )}(x_{1}^{n})=({1 \over \Gamma (\alpha )\beta ^{\alpha }})^{n}(\prod _{i=1}^{n}x_{i})^{\alpha -1}e^{{-1 \over \beta }\sum _{i=1}^{n}{x_{i}}}.\end{aligned}}$

Since $h(x_{1}^{n})$ does not depend on the parameter $(\alpha \,,\,\beta )$ and $g_{(\alpha \,,\,\beta )}(x_{1}^{n})$ depends only on $x_{1}^{n}$ through the function $T(X_{1}^{n})=(\prod _{i=1}^{n}{x_{i}},\sum _{i=1}^{n}{x_{i}})\,$,

the Fisher–Neyman factorization theorem implies $T(X_{1}^{n})=(\prod _{i=1}^{n}{x_{i}},\sum _{i=1}^{n}{x_{i}})\,$ is a sufficient statistic for $(\alpha \,,\,\beta )$.