Autoregressive (AR) Models

Autoregressive (AR) models form one of the fundamental components of the ARIMA family. The core idea behind an AR model is simple yet powerful: the current value of a time series can be predicted using a linear combination of its own previous values. This directly models the dependency of the series on its past, a concept we explored visually through autocorrelation plots in the previous chapter.

Definition and Structure

An Autoregressive model of order $p$, denoted as AR(p), predicts the current value $Y_t$ based on the $p$ preceding values ($Y_{t-1}, Y_{t-2}, \dots, Y_{t-p}$). The mathematical representation is:

$$Y_t = c + \phi_1 Y_{t-1} + \phi_2 Y_{t-2} + \dots + \phi_p Y_{t-p} + \epsilon_t$$

Let's break down the terms:

• $Y_t$: The value of the time series at the current time point $t$.
• $c$: A constant term (intercept).
• $\phi_1, \phi_2, \dots, \phi_p$: These are the autoregressive coefficients. They represent the weights applied to the past values. For instance, $\phi_1$ indicates how much the previous value ($Y_{t-1}$) influences the current value ($Y_t$).
• $Y_{t-1}, Y_{t-2}, \dots, Y_{t-p}$: The values of the time series at previous time points (lags 1, 2, ..., p).
• $p$: The order of the AR model, indicating how many past values are included in the regression.
• $\epsilon_t$: The error term at time $t$. This represents the part of $Y_t$ that is not explained by the past values. For a standard AR model, this term is assumed to be white noise, meaning it has a zero mean, constant variance, and is uncorrelated over time.

Intuition and Order Selection

Think of an AR(1) model ($p=1$). It states that the current value $Y_t$ is primarily dependent on the immediately preceding value $Y_{t-1}$, plus a constant and some random noise:

$$Y_t = c + \phi_1 Y_{t-1} + \epsilon_t$$

If $\phi_1$ is positive, a high value yesterday suggests a high value today. If it's negative, a high value yesterday suggests a low value today. An AR(2) model would use the two most recent past values ($Y_{t-1}$ and $Y_{t-2}$), and so on.

How do we determine the appropriate order $p$? As discussed in Chapter 3, the Partial Autocorrelation Function (PACF) plot is extremely useful here. For a pure AR(p) process, the PACF plot typically shows:

1. Significant correlations up to lag $p$.
2. A sharp cutoff after lag $p$, with subsequent partial autocorrelations falling close to zero (within the confidence interval).

Observing such a pattern in the PACF of your (stationary) time series suggests that an AR(p) model might be a good starting point.

A sample time series exhibiting AR(1) behavior. Notice how consecutive points tend to be relatively close in value, reflecting the influence of the previous point.

Stationarity Requirement

It's important to remember that standard AR models assume the time series $Y_t$ is stationary. Its mean, variance, and autocorrelation structure should not change over time. If your data is non-stationary (e.g., exhibits trends or seasonality), you'll typically need to transform it, often by differencing (as discussed in Chapter 2), before applying an AR model. This differencing step is the "Integrated" part represented by 'I' in ARIMA, which we will combine later in this chapter.

AR models capture one specific type of dependency structure in time series data. They form the basis for more complex models by explaining how past values influence the present. Next, we'll look at Moving Average (MA) models, which focus on the role of past forecast errors.