Managing Latent Variables in Discovery

A significant hurdle in discovering causal structure from observational data is the potential presence of unobserved common causes, often called latent variables or hidden confounders. Standard algorithms like the basic PC algorithm operate under the assumption of causal sufficiency, meaning that all relevant common causes of the measured variables are themselves measured. When this assumption is violated, which is common in systems, these algorithms can produce incorrect causal graphs, leading to spurious connections or missed causal links. Methods designed to perform causal discovery even when causal sufficiency cannot be guaranteed are presented.

The Challenge of Latent Variables

Consider a simple true causal structure where an unobserved variable $U$ confounds the relationship between measured variables $X$ and $Y$: $X \leftarrow U \rightarrow Y$. If $U$ is not measured, an algorithm assuming causal sufficiency might incorrectly infer a direct edge between $X$ and $Y$ (e.g., $X \rightarrow Y$ or $Y \rightarrow X$) based purely on the observed statistical dependence ($X \not\perp Y$). Similarly, if $X$ and $Y$ both cause $Z$, but are also confounded by $U$ ($X \leftarrow U \rightarrow Y$, $X \rightarrow Z$, $Y \rightarrow Z$), standard algorithms might struggle to orient the edges involving $Z$ correctly, especially if conditioning on $Z$ induces a dependency between $X$ and $Y$ (collider effect).

Managing latent variables in discovery involves detecting their potential influence and representing the resulting structural uncertainty, rather than necessarily identifying the exact latent variables themselves.

Constraint-Based Discovery with Latent Variables: The FCI Algorithm

The Fast Causal Inference (FCI) algorithm is a foundation constraint-based method explicitly designed to handle latent variables. Unlike PC, FCI does not assume causal sufficiency. It aims to infer properties of the causal structure that hold true across all possible underlying Directed Acyclic Graphs (DAGs) compatible with the observed conditional independencies, including those with latent confounders.

Partial Ancestral Graphs (PAGs)

Instead of outputting a single DAG or a Markov equivalence class of DAGs (like PC), FCI outputs a Partial Ancestral Graph (PAG). A PAG represents a class of DAGs, including those with latent variables, that are consistent with the data. PAGs use a richer set of edge markings to encode the uncertainty introduced by potential hidden confounders:

• $X \rightarrow Y$: A directed edge indicates that $X$ is a causal ancestor of $Y$ in all possible underlying DAGs compatible with the data.
• $X \leftrightarrow Y$: A bidirected edge indicates that $X$ and $Y$ are confounded by some latent variable in all compatible underlying DAGs.
• $X \text{ o}{-}\text{o } Y$: A non-directed edge with circles indicates uncertainty about the relationship. It could be $X \rightarrow Y$, $Y \rightarrow X$, or $X \leftrightarrow Y$ (confounding) in different compatible DAGs.
• $X \text{ o}\rightarrow Y$: A partially directed edge indicates that $Y$ is not an ancestor of $X$, but the relationship could be $X \rightarrow Y$ or $X \leftrightarrow Y$.

Example PAG with different edge types. $X \rightarrow Y$ denotes a definite causal ancestry. $Y \leftrightarrow Z$ (represented visually with a double arrow) indicates definite confounding. $Z \text{ o}\rightarrow W$ means $W$ is not an ancestor of $Z$, but $Z$ could cause $W$ or they could be confounded. $X \text{ o}{-}\text{o } W$ represents uncertainty about whether $X$ causes $W$, $W$ causes $X$, or they share a common confounder. The dashed node $L$ suggests a potential latent variable structure consistent with $Y \leftrightarrow Z$.

How FCI Works

FCI operates similarly to PC by iteratively testing conditional independencies. However, its rules for determining adjacencies and orientations are more complex to account for potential latent variables. It typically involves:

1. Adjacency Search: Finding pairs of variables that cannot be separated by any subset of other observed variables.
2. Collider Detection: Identifying potential colliders ($X \rightarrow Z \leftarrow Y$) based on specific independence patterns, but being cautious because latent variables can mimic these patterns (e.g., $X \leftrightarrow Z \leftrightarrow Y$).
3. Orientation Propagation: Applying a set of sophisticated orientation rules (derived from Spirtes, Glymour, and Scheines) that are sound even in the presence of latent variables. These rules use the identified adjacencies and potential colliders to orient edges as much as possible, resulting in the PAG markings.

Extensions like RFCI (Really Fast Causal Inference) and FCI+ aim to improve the computational efficiency and, in some cases, the accuracy of FCI, particularly for high-dimensional datasets.

Limitations and Interpretation of FCI/PAGs

While powerful, FCI comes with important considerations:

• Weaker Conclusions: The primary output is a PAG, representing a class of possible causal DAGs. This is inherently less specific than the output of algorithms assuming causal sufficiency. Extracting definite causal statements requires careful interpretation of the PAG edges.
• Reliance on CI Tests: Like all constraint-based methods, FCI's accuracy depends heavily on the reliability of the underlying conditional independence (CI) tests. Errors in CI testing (due to finite samples, non-linear relationships, non-Gaussian noise, etc.) can lead to errors in the resulting PAG. Sensitivity analysis regarding the significance threshold ($\alpha$) of the CI tests is often necessary.
• Computational Cost: FCI is generally more computationally expensive than PC, especially the orientation phase. Its complexity can scale poorly with the number of variables and the density of the graph. RFCI offers significant speedups but may sometimes be slightly less informative.

Score-Based Methods and Latent Variables

Standard score-based algorithms, such as Greedy Equivalence Search (GES), typically assume causal sufficiency. They search for the Markov equivalence class of DAGs that best fits the data according to a scoring criterion (e.g., BIC, BDeu). When latent confounders are present, the true causal structure might not be representable by any DAG over the observed variables only, potentially leading GES to converge to an incorrect structure.

While research exists on extending score-based methods to handle latent variables (e.g., by explicitly modeling latent variables or using scores effective against certain types of confounding), these approaches are often more complex and less generally applicable than FCI. Methods like Linear Non-Gaussian Acyclic Models (LiNGAM) can sometimes distinguish direct causation from confounding under specific assumptions (non-Gaussianity of error terms), but they don't provide a general solution for arbitrary latent variable structures.

Other Approaches

• Leveraging Diverse Data: As discussed in other sections, incorporating interventional data or exploiting heterogeneous datasets can sometimes provide additional information to resolve ambiguities caused by potential latent variables, potentially strengthening the conclusions drawn from algorithms like FCI.
• Identifying Latent Locations: Advanced research focuses on not just accounting for latents but trying to infer their presence between specific variables or even estimating their number. This often involves model comparison techniques or searching for specific independence patterns indicative of hidden variables.
• Proxy Variables: In some scenarios, if direct measurement of confounders is impossible, proxy variables (variables correlated with the latent confounders but not directly on the causal pathways of interest) can sometimes be used. Techniques like Proximal Causal Inference (covered in Chapter 4) leverage proxies for identification and estimation, representing a different approach than discovery algorithms like FCI.

Practical Implementation Notes

Python libraries such as causal-learn provide implementations of FCI, RFCI, and related algorithms. When using these tools:

• Pay close attention to the choice and configuration of the conditional independence test (e.g., Fisher's Z test for Gaussian data, Kernel-based CI tests like KCIT for non-linear relationships).
• Experiment with the significance level (alpha) for the CI tests, as the resulting PAG can be sensitive to this parameter.
• Be prepared for potentially long computation times on larger datasets.
• Focus on interpreting the PAG output correctly, acknowledging the uncertainty encoded in the edge markings.

Summary

Effectively managing latent variables is fundamental for trustworthy causal discovery from observational data. The assumption of causal sufficiency is often unrealistic. Constraint-based algorithms like FCI provide a principled way to infer causal structures while acknowledging the possible presence of hidden confounders, yielding Partial Ancestral Graphs (PAGs) that represent the inherent uncertainty. While standard score-based methods struggle with latents, and specialized methods exist for specific cases, FCI remains a standard approach for general discovery under potential confounding. Interpreting the resulting PAGs requires care, recognizing that they represent equivalence classes of causal models rather than a single definitive structure.