Frequency domain analysis

Background

$e^{i\theta} = cos \theta + i sin \theta$ Can use to prove basic trig identities and can invert to get $cos \theta = \frac{e^{i\theta}+e^{-i\theta}}{2}$ and $sin \theta = \frac{e^{i\theta} - e^{-i\theta}}{2}$

Sum of geometric series: $\frac{a(1 - r^n)}{1-r}$ .

General cosinusoid: $f(t) = a cos(\lambda t + \phi)$ , when t takes integer values makes sense to restrict $\lambda$ to $[0, 2\pi]$ , because of the aliasing problem

Frequency = Angular frequency / $2\pi$ , usually use angular for theory (no extra $2\pi$ s) and frequency for practice (more interpretable).

We will be considering siutations that are in some sense time invariant. One definition is $f(t + u) = f(t)$ but this too restrictive and the only solution is a constant. A less restrictive condition is $f(t+u) = C_u f(t)$ , which has general bounded solution of $f(t) = Ae^{i \lambda t}$ . Extends to sums of functions of this form.

Filters and filtering

A filter takes one time series and outputs another: $Y(t) = \mathcal{A}X(t)$ . A filter is linear if $ \mathcal{A} [\alpha X + \beta Y](t) = \alpha \mathcal{A}[X](t) + \beta \mathcal{A}[Y](t)$ , and time invariant if $\mathcal{A}[L_u X](t) = L^u \mathcal{A}[X](t)$ . An important class of linear, time-invariant filters can be written $\mathcal{A}[X](t) = \sum_{-\infinity}^\infinity a(u) X(t-u)$ where the $a(u)$ ’s are called filter coefficients.

Transfer functions

Filter coefficients provide complete description but may not provide much intuition about effect of the filter. Another way of investigating a filter is to look at its effects on complex exponentials. For convenience define $E^\lambda (t) = e^{i \lambda t}$ .

Clearly, $L^u E^\lambda (t) = e^{i \lambda (t+u)} = e^{i \lambda u} E^\lambda (t)$ and by linearity and time-invariance $\mathcal{A}[E^\lambda](t+u) = e^{i \lambda u}\mathcal{A}[E^\lambda](t)$ , setting $t=0$ gives $\mathcal{A}[E^\lambda](u) = e^{i \lambda u}\mathcal{A}[E^\lambda](0)$ . The function $A(\lambda) = \mathcal{A}[E^\lambda](u)$ called transfer function. Linear time-invariant filtering of complex exponential produces constant multiple with same frequency.

Note for any integer $k$ , $E^{\lambda + 2\pi k} = E^\lambda$ . In general transfer functions are complex-valued, and can often be useful to write in their polar form $A(\lambda) = G(\lambda)e^{i \phi(\lambda)}$ , where $G(\lambda)$ is the gain function, and $\phi(\lambda)$ the phase function.

Transfer function performs similarly on sines and cosines, and sums of complex exponential.

Computing transfer functions

If the filter coefficients satisfy $\sum_{-\infinity}^\infinity |a(u)| \lt \infinity$ , then the transfer function is given by $A(\lambda) = \sum_{-\infinity}^\infinity a(u) e^{-i \lambda u}$ . In mathetmatical terms, $A(\lambda)$ is the discrete Fourier transforms of the filter coefficients.

Sequential filtering

If $C[X] = A[B[X]](t)$ , then $C(\lambda) = A(\lambda)B(\lambda)$

Spectral theory

Suppose $X(t)$ is a stationary time-series with autocovariance function $\gamma(u)$ , which satisfies $\sum_{-\infinity}^\infinity |\gamma(u)| \lt \infinity$ , then we define the power spectrum of $X(t)$ to be:

\[f_XX (\lambda) = \frac{1}{2\pi} \sum_{-\infinity}^\infinity \gamma(u) e^{-i \lambda u}\]

Can be inverted to show:

\[\gamma (u) = \int^{2\pi}_0 e^{i \lambda t} f_XX (\lambda) d\lambda\]

This is the spectral decomposition of the autocovariance function. Since $\gamma(0) = var[X(t)]$ , $var[X(t)] = \int^{2\pi}_0 f_XX (\lambda) d\lambda$ , which shows the variability in $X(t)$ being broken down by frequency. To understand this we need to show that time series can be decomposed into independent frequency components.

The Cramér representation

Stieljes integral: $\int_a^b \g(x) dF(x) = lim \sum g(x_i) [F(x_{i+1}) - F(x_i)]$ . When F(x) is differentiable with $F'(x) = f(x)$ , reduces to $\int_a^b \g(x)f(x) dx$ . When F(x) is a step function with step of height $c_i$ at $u_i$ , reduces to $\sum_i c_i g(u_i)$ . Stieljes integral unifies both summation and integration.

To state the Cramér representation we need to extend the Steiljes integral to deal with stochastic processes.

A stochastic process, $Z(\lambda)$ on interval $[0, 2\pi]$ , is indexed set of random variables which assigns a random variable to each $\lambda \in [0, 2\pi]$ . Said to have uncorrelated increments if for $\lambda_1 \lt \lambda_2 \lt \lambda_3 \lt \lambda_4$ , $cor(Z(\lambda_2)-Z(\lambda_1), Z(\lambda_4) - \lambda_3)) = 0$ .

Possible to define an integral against a stochastic process:

\[ \int_0^{2\pi} \phi(\lambda) dZ(\lambda) = lim \sum_{n=0}^{N-1} \phi\left(\frac{2\pi n}{N}\right) \left[ Z\left( \frac{2\pi (n +1)}{N}\right) - Z \left( \frac{2\pi n}{N} \right) \right] \]

Where $Z_x(\lambda)$ is define as follows: