By model identification for a time series X we mean the process of selecting values of p,q so that the ARMA(p,q) process gives a reasonable fit to our data. The most important model identification tool is a plot of (an estimate of) the autocorrelation function of X; we use the abbreviation ACF for this function. Before we discuss doing this with real data we explore what plots of the ACF of various ARMA(p,q) plots should look like (in the absence of estimation error).
For an MA(p) process we found that
For an AR(1) process
the autocorrelation function is
To derive the autocovariance for a general AR(p) we mimic
the technique for p=1. If
then
for h > 0. Take these equations and divide through by CX(0)
and remember that
and
you see that the above recursions for
are p linear
equations in the p unknowns
.
They are
called the Yule Walker equations. For instance, when p=2 we get
which becomes, after division by CX(0)
It is possible to use generating functions to get explicit formulas for
the
but here we simply observe that we have two equations in
two unknowns to solve. The second equation shows that
Now look at ,
the characteristic polynomial, when a2=1 we have
Qualitative features: It is possible to prove that the solutions of these Yule-Walker equations decay to 0 at a geometric rate meaning that they satisfy for some . However, for general p they are not too simple.
If Z1,Z2 are iid
then we saw
If X and Y are jointly stationary then Z=aX+bY is
stationary and
In fact you can make AR processes which behave very much like
periodic processes. Consider the process
Here are graphs of trajectories and autocorrelations for a=0.3,0.6,0.9 and 0.99.
You should observe the slow decay
of the waves in the autocovariances, particularly for a near 1.
When a=1 the characteristic polynomial is 1-x+x2 which has roots
To get more insight consider the differential equation describing a
sine wave:
This is formalism only; there is no stationary solution of this equation. However, we see that AR(2) processes are at least analogous to the solutions of second order differential equations with added noise.
In order to identify suitable ARMA models using data we
need estimates of C and .
If we knew that we would see that