Asymptopics

Test statistics ($\theta \in R$ )

Likelihood ratio test

If appropriate assumptions hold, then $2[ l(\hat(\theta)_n, X) - l(\theta_0, X)] \rightarrow_D \chi^2_1$ .

Proof:

• expand $l(\theta, X)$ about $\hat{\theta}_n$ and evaluate at true parameter value
• $l(\theta_0) = l(\hat{\theta}_n) + l'(\hat{\theta}_n)(\theta_0 - \hat{\theta}_n + \frac{1}{2}l''(\theta^*_n)(\theta_0 - \hat{\theta}_n)^2$
• since $l'(\hat{\theta}_n) = 0$ , this gives $2[l(\hat{\theta} -l(\theta_0)] = -l''(\theta^*_n)(\theta_0 - \hat{\theta}_n)^2$
• $(\sqrt{n}(\theta_0 - \hat{\theta}_n))^2 \rightarrow_D (N(0, 1 / I_1(\theta_0))^2 = \chi^2_1 / I_1(\theta_0)$
• $l''(\theta^*_n)/n \rightarrow_p -I_1(\theta_0)$ since $\theta^*_n \rightarrow_p \theta_0$

For sufficiently large n, an approximate size $\alpha$ hypothesis test of $H_0: \theta = \theta_0$ is given by: Reject $H_0$ if $2[l(\hat{\theta}_n) - l(\theta)] \gt \chi^2_{1,\alpha}$

Wald score-test

Can replace $-l''(\theta^*_n)$ by other asymptotically equivalent values. Wald test replaces it with $I(\hat{\theta})$ (or sometimes with $-l''(\theta_0)$ or $I(\theta_0)$ ) to get $I(\hat{\theta}_n)(\theta_0 - \hat{\theta}_n)^2 \rightarrow_D \chi^2_1$ . Can be viewed as square of approximately standardised asymptotically normal random variables. In practice LR test is used most often.

Confidence intervals

Use the results the hypothesis tests. Eg. confidence interval will be all values of p, st. $l(p) \gt l(\hat{\theta}_n) - \chi^2_{1,\alpha} / 2$

Test statistics ($\theta \in R^s$ )

Extending results to multiparameter case is reasonably straightforward, but very tedious. Results follow.

Asymptotic normality and efficiency of $\hat{\theta}_n \in R^s$ . Exactly the same as for single parameter, but uses vectors. The delta theorem extends similarly.

Simple hypothesis

Want to test if $\theta$ is a particular value. $H_0: \theta = \theta_0$ . Likelihood and Wald score tests extend in the obvious way.

Composite hypothesis

$H_0: \theta \in \Theta_0 \in \Theta$ , where $\Theta_0$ is a $s -r$ dimensional subset of $\Theta$ . We shall assume (reparameterising if necessary) that $\theta = (\psi \lambda) ^T $ , where $\psi \in \real^r$ , $\lambda \in \real^{s-r}$ , and $\theta \in \Theta_0$ are all points in $\Theta$ for which $\psi$ equals some specified value.

Likelihood: $2[l(\hat{\theta}_n) - l(\hat{\theta}_{0n})] \rightarrow_D \chi^2_r$ , where $\hat{\theta}_{0n}$ is the MLE in $\Theta_0$ .

Wald: $(\hat{\psi}_n - \psi_0)^T [[I^{-1}(\hat{\theta}_n)]_{\psi\psi}]^{-1}(\hat{\psi}_n - \psi_0) \rightarrow_D \chi^2_r$ , where $[I^{-1}(\hat{\theta}_n)]_{\psi\psi}$ is the upper $r \times r$ submatrix of $I^{-1}$ , called Fisher information for $\psi$ in presence of nuisance parameter $\lambda$ .