Xappppp

A proof of Maximum Likelihood Estimator

1. Proving Convergence in Probability

if set $\,\,\,S_{n}(\theta )={\frac {1}{n}}{\frac {\partial logl(\theta )}{\partial \theta }}\,\,\,$, then usually we will have following conditions:

$ES_{n}(\theta )=0,\,\,\,\,\,\,\,-S'_{n}(\theta ^{})>0$

and we know

$P(|\theta _{mle}-\theta |\geq \delta )=P(\theta _{mle}\geq \theta +\delta )+P(\theta _{mle}\leq \theta -\delta )$

so

${\begin{aligned}P(\theta _{mle}\geq \theta +\delta )&=P(0\leq S_{n}(\theta +\delta ))\\&=P(0\leq S_{n}(\theta )+\delta S'_{n}(\theta ^{*}))\\&=P(S_{n}(\theta )\geq -\delta S'_{n}(\theta ^{*}))\\&\leq P(S_{n}(\theta )>0)\rightarrow 0\,\,\,\,\,\,\,(\,\,\,since\,\,\,S_{n}(\theta )\rightarrow _{a.s.}0\,\,)\end{aligned}}$

similarly

$P(\theta _{mle}\leq \theta -\delta )\rightarrow 0$

thus complete the proof of

$\theta _{mle}\rightarrow _{p}\theta$

2. Derive limiting distribution

given $\theta _{mle}\rightarrow _{p}\theta$

we will have

$S_{n}(\theta _{mle})=0=S_{n}(\theta _{})+(\theta -\theta _{mle})S'_{n}(\theta _{})+o_{p}(\theta -\theta _{mle})$

which indicates

${\sqrt {n}}(\theta _{mle}-\theta _{})={\frac {{\sqrt {n}}S_{n}(\theta _{})}{S'_{n}(\theta _{})+o_{p}(1)}}\rightarrow {\frac {N(0,I_{1}[\theta ])}{I_{1}[\theta ]}}=N(1,I_{1}^{-1}[\theta ])$