Estimation and identification

ACF

Do not usually know $\gamma(u)$ etc. and must estimate from data, with $\hat{y} (u) = \frac{1}{T} \sum (Y_{t+u} - \bar{Y})(Y_t - \bar{Y})$ . Estimator is biased (using $1 / T$ instead of $1 / T-U$ ) because gives positive definite $\hat{\gamma}(u)$ which ensures that all covariances are positive and has lower MSE for time series where $\gamma(u) \to 0$ as $u \to \infinity$ . Estimated autocorrelation function $r(u) = \hat{\gamma}(u) / \hat{\gamma}(0)$ .

To estimate confidence intervals etc. need some distribution theory (from Andersen, 1971). If $Y_t = \mu + \sum \psi_u \epsilon_{t-u}$ , $\sum |\psi_u| \lt \infinity$ , and $\sum u \psi_u^2 \lt \infinity$ , then $r(u) ~ AN(0, c_uu / T)$ , $cor(r(u), r(v)) = c_uv / \sqrt{c_uu c_vv}$ , where $c_uv$ is given by Bartletts formula, $\sum_{k=0}^\infinity (\rho(k - u) + \rho(k + u) - 2\rho(k)\rho(u))(\rho(k - u) + \rho(k + u) - 2\rho(k)\rho(v))$ .

White noise

If $Y_t$ are iid: $var(r(u)) \approx 1 / T$ , $cor(r(u), r(v)) \approx 0$ .

AR (1) series

In this case $\rho(u) = \phi^2$ for $u \gt 0$ , so $var(r(1)) = \frac{1}{T}(1 - \phi^2) $ , and $var(r(u)) = \frac{1}{T}\frac{1 + \phi^2}{1-\phi^2}$ . When $|\phi|$ near 1, accurate at low $u$ , inaccurate at high.

General MA (q) series

“Easy” to see that: $c_uu = 1 + 2 \sum_{v=1}^q \rho(v)^2$ for $u \gt q$ .

PACF

Use Derbin-Levinson recursion on estimated ACF.

For CI(confidence interval)s use Quenouille (1949) which states that if true model is AR (p) then when $u \gt p$ , $\hat{\phi}(u) ~ AN(0, 1 / T)$ , and so we can use $\pm 2 / \sqrt{T}$ as 95% CIs.

System identification

Using above results:

• ACF: decay, PACF: decay = ARMA (p = ?, q = ?)
• ACF: decay, PACF: cutoff = AR (p = cutoff)
• ACF: cuttoff, PACF: decay = MA (q = cutoff)
• ACF: cuffoff, PACF: cutoff = Problem!

Very slow decay in the ACF indicates non-stationarity