Example from last week lectures

Example: Let \(X_1,...,X_n\) be iid \(uniform(0,\theta)\).

\[ f(x)\begin{cases}{1\over \theta}~~~0\leq x\leq \theta,\\0,~~~elsewhere\end{cases} \]

  1. Find the method of moments estimator of \(\theta\).

k=1, only need the first moment

\[ \mu_1=E(X_1)={\theta\over 2}\\ \hat\mu_1={1\over n}\sum_{i=1}^nX_i \]

Setting \(\mu_1=\hat \mu_1\), we have \({\theta\over 2}={1\over n}\sum_{i=1}^nX_i\Rightarrow\hat\theta=2\bar X\)

  1. Find the MLE of \(\theta\).

\[ lik(\theta)=\prod^n_{i=1}f(x_i|\theta)=\begin{cases}{1\over\theta^n},~~~0\leq X_1,...,X_n\leq \theta\\0,~~~ elsewhere\end{cases} \]

Introduction to Confidence Intervals

We can use the sample mean \(\bar X\) as a point estimate for the population mean \(\mu\).

use only \(\bar X\)to estimate \(\mu\), it is not very informative since it does not quantify accuracy.

A confidence interval is an interval for which we can asserts, with a given degree of confidence/certainty, that it includes the true value of the parameter being estimated.

Z Confidence Intervals

  • Confidence intervals that use percentiles from the standard normal distribution.

\[ z_\alpha=100(1-\alpha)-th~percentile \]

\(z_\alpha\) is also called as upper percentile sometime.

For example:

\[ \alpha =0.025\\ z_\alpha=qnorm(1-\alpha)=1.96 \]

Let \(X_1,...,X_n\) be iid random variables with parameter \(\theta\).

  • By CLT, many estimators \(\hat \theta\) of \(\theta\) will be approximately normally distributed.

\[ \hat\theta\sim N(\theta,\sigma_{\hat\theta}^2), approximately\\ {\hat\theta-\theta\over \sigma_{\hat\theta}}\sim N(0,1), approximately \]

you can also replace \(\sigma_{\hat\theta}\) by \(S_{\hat\theta}\) (estimated standard error) if \(\sigma_{\hat\theta}\) is unknown

  • This implies:

\[ P(-1.96<{\hat\theta-\theta\over \sigma_{\hat\theta}}<1.96)=0.95 \]

  • This results in a 95% confidence interval for \(\theta\):

\[ \hat \theta-1.96S_{\hat \theta}\leq \theta\leq \hat\theta+1.96S_{\hat \theta} \]

\[ {\hat\theta-\theta\over \sigma_{\hat\theta}}\leq1.96 \Leftrightarrow \hat\theta-\theta\leq1.96\sigma_{\hat\theta} \Leftrightarrow \theta\geq\hat \theta-1.96\sigma_{\hat\theta} \]

\[ {\hat\theta-\theta\over \sigma_{\hat\theta}}\geq-1.96 \Leftrightarrow \hat\theta-\theta\geq -1.96\sigma_{\hat\theta} \Leftrightarrow \theta\leq\hat \theta+1.96\sigma_{\hat\theta} \]

you can replace \(\sigma_{\hat\theta}\) by \(S_{\hat \theta}\)

  • The general \((1-\alpha)100%\) confidence interval for \(\theta\) is

\[ \hat\theta-z_{\alpha/2}S_{\hat\theta}\leq\theta\leq\hat\theta+z_{\alpha/2}S_{\hat\theta} \]

gives us a range of possible values of the true parameter \(\theta\), such that we are \((1-\alpha)100%\) confident that it contains the true \(\theta\)

Common choice of \(\alpha\): 0.01 (99%), 0.05 (95%), 0.01 (90%).

  • Z confidence interval will generally be used for proportions \((\theta=p)\).

only works well if \(\sigma\) is known in mean estimation or sample size is very large.

T Confidence Intervals

  • Confidence intervals that use percentiles from the T distribution.

  • \(T_v\) is a random variable from the T distribution with v degrees of freedom. (or df)

degree of freedom often depends on the sample size.

\[ \lim_{v\to+\infty}t_v\sim N(0,1) \]

  • In R:

  • dt(x, v) gives the PDF of \(T_v\) for x

  • pt(x, v) gives the CDF of \(T_v\) for x

  • qt(s, v) gives the s100 percentile of \(T_v\)

  • rt(n, v) generates a random sample of n \(T_v\) random variables

  • Many estimator \(\hat\theta\) of \(\theta\) satisfy

\[ {\hat\theta-\theta\over S_{\hat\theta}}\sim T_v \]

when you replace the estimated by \(S_{\hat \theta}\), the result shall have greater variability.

\({\hat\theta-\theta\over S_{\hat\theta}}\) has a greater variability than \({\hat\theta-\theta\over \sigma_{\hat\theta}}\)

  • This gives a \((1-\alpha)100\%\) confidence interval for \(\theta\):

\[ (\hat\theta-t_{v,\alpha/2}S_{\hat\theta},\hat\theta+t_{v,\alpha/2}S_{\hat\theta}) \]

Notes:

  1. T confidence intervals will generally be used for means.

  2. \(t_{v,\alpha/2}\to z_{\alpha/2}\) as \(n\to \infty\)

  3. Common values of \(\alpha\) are 0.1,0.05,and 0.01