Model identification summary: The simplest model identification tactic is to look for either a pure MA or pure AR model. To do so:
If X is not stationary will need to transform X to find a related stationary series. We will consider in this course two sorts of non-stationarity -- non constant mean and integration.
Non constant mean: If is not constant we will hope to model using a small number of parameters and then model as a stationary series. Three common structures for are linear, polynomial and periodic:
Linear trend: Suppose
Method 1: regression (detrending). We regress
Xt on t to get
and
and analyze
Method 2: differencing. Define
Then
Since this is a linear filter applied to the series
Y it is stationary if Y is. BUT, it might be stationary
even if Y is not. Suppose that
is an iid mean 0
sequence. Then
These random walk models are common in Economics. In physics they are used in the limit of very small time increments - this leads to Brownian motion.
Definition: X satisfies an ARIMA(p,d,q) model if
Remark: If where Y is stationary and is a polynomial of degree less than or equal to p then is stationary. (So a cubic shaped trend could be removed by differencing 3 times.)
WARNING: it is a common mistake in students' data analyses to over difference. When you difference a stationary ARMA(p,q) you introduce a unit root in the defining polynomial - the result cannot be written as an infinite order moving average.
Detrending: Define a response vector
The problem in our context (it is almost always a problem) is that you can only use is you know . In our context you won't know until you have removed a trend, selected a suitable ARMA model and estimated the parameters. The natural proposal is to follow an iterative process:
The process is repeated until the estimates stop changing in any important way.
Folklore: There is evidence that the OLS estimator has a variance which is not too much different from GLS in common ARMA models.
Every winter the measured (not reported) unemployment rate
in Canada rises. A simple model which has this feature has
a non-stationary mean of the form
Definition: Deseasonalization is the process of transforming Xto eliminate this sort of seasonal variation in the mean.
Method 1: Regression. Estimate
Method B: Seasonal differencing:
Definition: A multiplicative
model has the form:
As an example consider the model
Fitting the I part is easy we simply difference d times. The same observation applies to seasonal multiplicative model. Thus to fit an ARIMA(p,d,q) model to X you compute Y =(I-B)d X (shortening your data set by d observations) and then you fit an ARMA(p,q) model to Y. So we assume that d=0.
Simplest case: fitting the AR(1) model
Our basic strategy will be:
Generally the full likelihood is rather complicated; we will use conditional likelihoods and ad hoc estimates of some parameters to simplify the situation.
If the errors
are normal then so is the series X. In general
the vector
has a
where
and
is a vector all of whose entries are
.
The joint density of X is
It is possible to carry out full maximum likelihood by maximizing the quantity in question numerically. In general this is hard, however.
Here I indicate some standard tactics. In your homework I will be asking you to carry through this analysis for one particular model.
Consider the model
Now compute
To find you now plug and into (getting the so called profile likelihood ) and maximize over . Having thus found the mles of and are simply and .
It is worth observing that fitted residuals can then be calculated: