4 like 0 dislike
48 views
How do I prove the Central Limit Theorem for Sample Means?
solved
| 48 views

1 like 0 dislike

Proof of the Central Limit Theorem

Suppose $X_{1}, \ldots, X_{n}$ are i.i.d. random variables with mean 0 , variance $\sigma_{x}^{2}$ and Moment Generating Function (MGF) $M_{x}(t)$. Note that this assumes an MGF exists, which is not true of all random variables.
Let $S_{n}=\sum_{i=1}^{n} X_{i}$ and $Z_{n}=S_{n} / \sqrt{n \sigma_{x}^{2}}$. Then
$M_{S_{n}}(t)=\left(M_{x}(t)\right)^{n} \text { and } M_{Z_{n}}(t)=\left(M_{x}\left(\frac{t}{\sigma_{x} \sqrt{n}}\right)\right)^{n} .$
Using Taylor's theorem, we can write $M_{x}(s)$ as
$M_{x}(s)=M_{x}(0)+s M_{x}^{\prime}(0)+\frac{1}{2} s^{2} M_{x}^{\prime \prime}(0)+e_{s},$
where $e_{s} / s^{2} \rightarrow 0$ as $s \rightarrow 0$.
$M_{x}(0)=1$, by definition, and with $E\left(X_{i}\right)=0$ and $\operatorname{Var}\left(X_{i}\right)=\sigma_{x}^{2}$, we know $M_{x}^{\prime}(0)=0$ and $M_{x}^{\prime \prime}(0)=\sigma_{x}^{2}$. So
$M_{x}(s)=1+\frac{\sigma_{x}^{2}}{2} s^{2}+e_{s} .$
Letting $s=t /\left(\sigma_{x} \sqrt{n}\right)$, we have $s \rightarrow 0$ as $n \rightarrow \infty$, and
$M_{Z_{n}}(t)=\left(1+\frac{\sigma_{x}^{2}}{2}\left(\frac{t}{\sigma_{x} \sqrt{n}}\right)^{2}+e_{n}\right)^{n}=\left(1+\frac{t^{2}}{2 n}+e_{n}\right)^{n}$
where $n \sigma_{x}^{2} e_{n} / t^{2} \rightarrow 0$ as $n \rightarrow \infty$.
If $a_{n} \rightarrow a$ as $n \rightarrow \infty$, it can be shown that
$\lim _{n \rightarrow \infty}\left(1+\frac{a_{n}}{n}\right)^{n}=e^{a} .$
It follows that
$\lim _{n \rightarrow \infty} M_{Z_{n}}(t)=\lim _{n \rightarrow \infty}\left(1+\frac{t^{2} / 2+n e_{n}}{n}\right)^{n}=e^{t^{2} / 2},$
which is the MGF of a standard Normal. If the MGF exists, then it uniquely defines the distribution. Convergence in MGF implies that $Z_{n}$ converges in distribution to $N(0,1)$.
The practical application of this theorem is that, for large $n$, if $Y_{1}, \ldots, Y_{n}$ are independent with mean $\mu_{y}$ and variance $\sigma_{y}^{2}$, then
$\sum_{i=1}^{n}\left(\frac{Y_{i}-\mu_{y}}{\sigma_{y} \sqrt{n}}\right) \dot{\sim} N(0,1), \quad \text { or } \quad \bar{Y} \dot{\sim} N\left(\mu_{y}, \sigma_{y}^{2} / n\right) .$
How large is "large" depends on the distribution of the $Y_{i}$ 's. If Normal, then $n=1$ is large enough. As the distribution becomes less Normal, larger values of $n$ are needed.

by Wooden Status (264 points)
selected by

2 like 0 dislike
2 like 0 dislike
2 like 0 dislike
1 like 0 dislike
1 like 0 dislike
0 like 0 dislike
0 like 0 dislike
0 like 0 dislike
1 like 0 dislike