Granger Causality: Formulation and Shortcomings

Analyzing causality in temporal data requires specialized approaches. One of the earliest and most widely known concepts for analyzing relationships between time series is Granger causality. While influential, especially in econometrics, it's essential for practitioners in machine learning systems to understand its precise definition and, more importantly, its significant limitations when interpreting results as evidence of genuine causal influence.

Granger Causality: The Concept of Predictive Influence

Developed by Nobel laureate Sir Clive Granger, the core idea isn't about structural causation in the way we understand it from Structural Causal Models (SCMs). Instead, Granger causality is fundamentally a statement about predictability. A time series $X_t$ is said to "Granger-cause" another time series $Y_t$ if past values of $X_t$ contain information that helps predict $Y_t$ more effectively than the information already contained in the past values of $Y_t$ itself.

Formal Definition

Let $Y_t$ and $X_t$ be two stationary time series. Consider two autoregressive models for predicting $Y_t$:

1. Restricted Model: Predicts $Y_t$ using only its own past values (lags).

   $$Y_t = \alpha_0 + \sum_{i=1}^{p} \alpha_i Y_{t-i} + \epsilon_{1,t}$$

   Here, $p$ is the number of lags included, $\alpha_i$ are coefficients, and $\epsilon_{1,t}$ is the prediction error (residual) for this model.

2. Unrestricted Model: Predicts $Y_t$ using past values of both $Y_t$ and $X_t$.

   $$Y_t = \beta_0 + \sum_{i=1}^{p} \beta_i Y_{t-i} + \sum_{j=1}^{q} \gamma_j X_{t-j} + \epsilon_{2,t}$$

   Here, $q$ is the number of lags for $X_t$, $\beta_i$ and $\gamma_j$ are coefficients, and $\epsilon_{2,t}$ is the prediction error for this model.

The definition of Granger causality depends on comparing the variances of the prediction errors ($\epsilon_{1,t}$ and $\epsilon_{2,t}$) from these two models.

Definition: $X_t$ Granger-causes $Y_t$ if the variance of the prediction error $\epsilon_{2,t}$ from the unrestricted model is statistically significantly smaller than the variance of the error $\epsilon_{1,t}$ from the restricted model. In simpler terms, adding past $X_t$ values significantly improves the prediction of $Y_t$. This is typically tested using an F-test on the joint significance of the coefficients $\gamma_j$ (i.e., $H_0: \gamma_1 = \gamma_2 = ... = \gamma_q = 0$).

Similarly, we can test if $Y_t$ Granger-causes $X_t$ by swapping the roles of $X_t$ and $Y_t$ in the models above.

Significant Shortcomings and Misinterpretations

While the concept of improved predictability is useful, equating Granger causality with structural causation (i.e., asserting that manipulating $X$ would lead to a change in $Y$) is fraught with peril. For advanced practitioners building reliable systems, recognizing these limitations is absolutely necessary.

1. Predictability vs. Causation: This is the most fundamental limitation. Granger causality only establishes whether past $X$ helps predict future $Y$. It does not imply that $X$ exerts a physical or structural influence on $Y$. Correlation, even lagged correlation, does not equal causation.

2. Unobserved Confounding: The most common reason for spurious Granger causality is the presence of an unobserved common cause $Z_t$ that influences both $X_t$ and $Y_t$ with different time lags. If $Z_t$ affects $X_t$ first and then $Y_t$, the past values of $X_t$ will appear predictive of $Y_t$ simply because they act as a proxy for the influence of the unobserved $Z_t$. Standard Granger tests do not account for such confounders.

   An unobserved common cause $Z_t$ influencing both $X_t$ and $Y_t$ at different lags can create spurious Granger causality from $X_t$ to $Y_t$.

3. Instantaneous Effects: Granger causality is defined based on past values. It cannot detect contemporaneous causal relationships where $X_t$ influences $Y_t$ within the same time period ($t$). These effects are absorbed into the error terms' correlation.

4. Omitted Variables: The test assumes that all relevant predictive information is contained within the past values of $X_t$ and $Y_t$. If a third observed variable, $W_t$, influences $Y_t$ and is also correlated with past $X_t$, omitting $W_t$ from the models can lead to incorrect conclusions about the relationship between $X_t$ and $Y_t$. This necessitates multivariate extensions (Vector Autoregression, VAR) for practical use, but even VAR-based Granger tests suffer from confounding if relevant variables are omitted.

5. Non-Linear Relationships: The standard formulation relies on linear autoregressive models. If the true relationship between $X_t$ and $Y_t$ is non-linear, the linear Granger test might fail to detect predictive power, even if a causal link exists. Non-linear extensions exist but come with their own complexities.

6. Non-Stationarity: The statistical tests underpinning Granger causality typically assume that the time series $X_t$ and $Y_t$ are (covariance) stationary. Applying the tests to non-stationary data (e.g., series with trends or unit roots) can produce spurious results indicating Granger causality where none exists. Pre-processing steps like differencing are often required, but these can also alter the underlying relationships.

7. Measurement Error: Errors in measuring $X_t$ or $Y_t$ can bias the coefficient estimates and affect the test outcomes, potentially masking true relationships or suggesting spurious ones.

Context and Applicability

Despite these serious limitations from a structural causal inference perspective, Granger causality tests can sometimes serve as a preliminary exploratory tool in time series analysis. They can help identify potential lagged predictive relationships that warrant further, more rigorous causal investigation using methods discussed later in this chapter, such as Structural Vector Autoregression (SVAR) or time-series causal discovery algorithms. However, interpreting a significant Granger causality result as definitive proof of a causal link is a common mistake that should be strictly avoided in sophisticated ML system development.

Understanding Granger causality provides a foundation for appreciating why more advanced techniques are needed for causal inference in temporal settings. We now turn our attention to methods like SVAR, which attempt to impose more structure to identify causal effects, albeit with their own sets of assumptions.