Postscript version of these notes

STAT 804

Lecture 22 Notes

Properties of the Periodogram

The discrete Fourier transform

$${\hat X}(\omega) = \frac{1}{\sqrt{T}}\sum_{t=0}^{T-1} X_t \exp(2\pi \omega t i)$$

is periodic with period 1 because all the exponentials have period 1. Moreover,

$${\hat X}(1-\omega) = \frac{1}{\sqrt{T}}\sum_{t=0}^{T-1} X_t \... ...\pi \omega t i) \exp(2\pi t i) = {\overline{{\hat X}(\omega)}}$$

so that the periodogram satisfies

$$\vert{\hat X}(1-\omega)\vert^2 = \vert{\hat X}(\omega)\vert^2 \,$$

Thus the periodogram is symmetric around $\omega=1/2$ which is called the Nyquist or folding frequency. (The value is always 1/2 in cycles per point but usually it would be converted to cycles per time unit like year or day or whatever.)

Similarly the power spectral density f_X given by

$$f_X(\omega) = \sum_{-\infty}^\infty C_X(h) \exp(2 \pi h \omega i)$$

is periodic with period 1 and satisfies

$$f_X(-\omega) = f_X(\omega)$$

which is equivalent to

$$f_X(1-\omega) = f_X(\omega) \, .$$

Spectra of Some Basic Processes

Here I compute the spectra of a few basic processes directly and then indirectly by a more powerful technique.

Direct from the definition

White Noise:

Since $C_\epsilon(k)=0$ for all non-zero k we have

$$f_\epsilon(\omega) \equiv C_\epsilon(0) = \sigma^2_\epsilon \, .$$

MA(1):

For $X_t = \epsilon_t - b \epsilon_{t-1}$ we have $C_X(0)= \sigma^2(1+b^2)$ and $C_X(\pm 1) = -b\sigma^2$ so that

$$\begin{eqnarray*}f_X(\omega)& =& \sigma^2(1+b^2)-b\sigma^2 (\exp(2\pi\omega i) +... ...\omega i))\\ & = & \sigma^2(1+b^2)-2b\sigma^2 \cos(2\pi\omega) \end{eqnarray*}$$

AR(1):

We have $C_X(k) = \rho^{\vert k\vert} C_X(0)$ and

$$\begin{eqnarray*}f_X(\omega)& =& C_X(0)( 1+\sum_{k>0} \rho^k(\exp(2\pi\omega ki)... ... & = & \frac{\sigma_\epsilon^2}{1+\rho^2 -2\rho\cos(2\pi\omega)} \end{eqnarray*}$$

Using filters

Any mean 0 ARMA(p,q) process can be rewritten in MA form as

$$X_t = \sum_{s=0}^\infty a_s \epsilon_{t-s}$$

and then the covariance of X is

$$C_X(h) = \sum_{r=0}^\infty \sum_{s=0}^\infty a_r a_s Cov(\epsilon_{t+h-r}, \epsilon_{t-s})$$

Although the covariance simplifies for white noise, let us simply write C(t+h-r -(t-s)) = C(h+s-r) for the covariance in this double sum so that the calculation will apply to any stationary $\epsilon$. Then plug this double sum into the definition of f_X to get

$$f_X(\omega) = \sum_{h=-\infty}^\infty \exp(2\pi\omega h i) \sum_{r=0}^\infty \sum_{s=0}^\infty a_r a_s C(h+s-r) \\$$

Now write the h in the complex exponential in the form (h+s-r) +r-sand bring the sum over h to the inside to get

$$f_X(\omega) = \sum_{r=0}^\infty a_r e^{2\pi\omega ri} \sum_{s... ... si} \sum_{h=-\infty}^\infty \exp(2\pi\omega(h+s-r)i) C(h+s-r)$$

Finally make the substitution k=h+s-r in the inside sum and define

$$A(\omega) = \sum_{r=0}^\infty a_r e^{2\pi\omega ri}$$

to see that

$$f_X(\omega) = A(\omega) {\overline{A(\omega)}} f_\epsilon(\omega)$$

or

$$f_X(\omega) = \vert A(\omega)\vert^2 f_\epsilon(\omega) \, .$$

The function A (or $\bar A$) is called the frequency response function and |A|² the power transfer function. The jargon gain is sometimes used for |A|.

The Spectrum of an ARMA(p,q)

An ARMA(p,q) process X satisfies

$$\sum_{s=0}^p a_s X_{t-s} = \sum_{r=0}^q b_r \epsilon_{t-r}$$

so that if Y is the process $\sum_{s=0}^p a_s X_{t-s}$ then

$$f_Y(\omega) = \vert A(\omega)\vert^2 f_X(\omega)$$

where $A(\omega) = \sum_{s=0}^p a_s \exp(2\pi\omega si)$. At the same time $Y_t = \sum_{r=0}^q b_r \epsilon_{t-r}$ so

$$f_Y(\omega) = \vert B(\omega)\vert^2 f_\epsilon(\omega)$$

where $B(\omega) = \sum_{s=0}^q b_s \exp(2\pi\omega si)$. Hence

$$f_X(\omega) = \frac{\vert B(\omega)\vert^2}{\vert A(\omega)\vert^2}\, .$$

For example, in the ARMA(1,1) case $X_t-aX_{t-1} = \epsilon_t-b\epsilon_{t-1}$we find (referring to our MA(1) calculation above that $\vert B(\omega)\vert^2 = 1+b^2-2b \cos(2\pi\omega)$ and $\vert A(\omega)\vert^2 = 1+a^2-2a \cos(2\pi\omega)$ so that

$$f_X(\omega) = \frac{1+b^2-2b \cos(2\pi\omega)}{1+a^2-2a \cos(2\pi\omega} \, .$$