Causal Discovery from Heterogeneous Data

"Data rarely arrives in a single, neat package governed by one fixed probability distribution. More often, we encounter heterogeneous data: collections of datasets originating from different sources, conditions, environments, or experimental setups. For instance, you might have sensor readings from different factories, patient data from multiple hospitals, or economic indicators across various regions or time periods. While the underlying causal mechanisms might be shared, the specific distributions of variables, the interventions performed, or even the variables measured can differ significantly."

Naively pooling heterogeneous datasets and applying standard causal discovery algorithms discussed earlier (like PC, FCI, or GES) is often problematic. Differences in distributions across datasets can violate the assumptions these algorithms rely upon, particularly the faithfulness assumption. Combining data inappropriately can lead to spurious edges or missed connections, effectively masking the true underlying causal structure. Imagine combining data where a confounder $Z$ has different relationships with $X$ and $Y$ in two subgroups; pooling might obscure or even invert the apparent relationship between $X$ and $Y$.

Turning Heterogeneity into an Advantage

Instead of viewing heterogeneity solely as a complication, advanced causal discovery techniques treat it as a valuable source of information. The variations across datasets, if modeled correctly, can provide constraints that help identify causal relationships that would be ambiguous or undiscoverable from any single dataset alone. The central idea is that while observational distributions might change across environments, the underlying causal mechanisms themselves are often invariant or stable.

Consider a simple example: Suppose we have two datasets measuring variables $X, Y, Z$. In dataset 1, the variance of $X$ is low, while in dataset 2, it's high, perhaps due to different background conditions or interventions targeting $X$. If the relationship $X \rightarrow Y$ is truly causal, we expect the mechanism generating $Y$ from $X$ (and its other parents) to remain consistent across both datasets, even though the distribution of $X$ changes. Conversely, if $Y$ were the cause of $X$, changes in $X$'s distribution shouldn't affect the $Y \rightarrow X$ mechanism itself. By searching for structural models or relationships that remain stable across these differing environments, we can gain stronger evidence for specific causal links.

Invariant Causal Prediction (ICP)

A prominent framework exploiting this principle is Invariant Causal Prediction (ICP). ICP aims to identify the set of direct causes of a target variable $Y$ by leveraging data from multiple environments or settings. The core assumptions are:

1. Existence of Environments: The data can be partitioned into distinct environments $\mathcal{E} = \{e_1, e_2, ..., e_m\}$, where $m \ge 2$. These environments could represent different experimental conditions, time periods, locations, or known interventions.
2. Underlying SCM Stability: There exists an underlying Structural Causal Model (SCM) generating the variables, and the conditional distribution of $Y$ given its direct causes $Pa(Y)$ is the same across all environments $e \in \mathcal{E}$. That is, $P(Y | Pa(Y))$ is invariant.
3. Heterogeneity: The distributions of variables other than $Y$ given its parents, or interventions, can differ across environments. This variation is what provides identifying information.

ICP proceeds by testing hypotheses. For a given candidate set of predictors $S$, it tests the null hypothesis that $S$ contains the true causal parents $Pa(Y)$ and leads to an invariant prediction model across all environments. This typically involves performing regression of $Y$ on $S$ within each environment and testing if the coefficients and residual distributions are statistically identical.

The final output of ICP is the intersection of all sets $S$ that cannot be rejected. Under its assumptions, this intersection is guaranteed to contain only true causal parents of $Y$, providing a potentially conservative but highly reliable set of causal predictors.

Extending Discovery Algorithms for Heterogeneity

Other methods adapt broader discovery algorithms:

• Joint Causal Inference (JCI): Frameworks under JCI attempt to leverage various data types (observational, interventional, data with differing distributions) simultaneously to estimate a single underlying causal graph.
• Environment-Specific Constraint/Score-Based Methods: Some algorithms modify constraint-based (like FCI variants) or score-based methods (like GES variants) to explicitly account for multiple environments. They might search for structures that are consistent across environments or use the environmental information to resolve ambiguities (e.g., orienting edges). For example, observing that $X$ and $Y$ are independent in an environment where $Z$ is intervened upon, but dependent otherwise, provides strong evidence for specific pathways involving $Z$.

Example: Heterogeneity Resolving Ambiguity

Consider three variables $X, Y, Z$. From purely observational data in one environment, constraint-based methods might identify the equivalence class containing $X \rightarrow Y \rightarrow Z$, $X \leftarrow Y \rightarrow Z$, and $X \leftarrow Y \leftarrow Z$. Now, suppose we have a second dataset from a different environment where the variance of $X$ is significantly increased due to some external factor, but the conditional distributions $P(Y|X)$ and $P(Z|Y)$ appear unchanged compared to the first environment. This invariance, coupled with the change in $X$'s distribution propagating to $Y$ and $Z$, strongly favors the structure $X \rightarrow Y \rightarrow Z$.

The diagram illustrates how observing consistent relationships (bold arrows) across environments where $X$'s properties differ (highlighted node) supports the $X \rightarrow Y \rightarrow Z$ causal chain.

Practical Implementation and Approach

When working with heterogeneous data for causal discovery:

1. Data Partitioning: Carefully define the distinct environments based on known interventions, different time periods, experimental conditions, or subgroups. The validity of the environmental partitioning is significant.
2. Assumption Validation: The assumption of invariant causal mechanisms is strong. Sensitivity analyses or domain knowledge should be used to assess its plausibility. Are there reasons to believe the causal laws themselves change between environments?
3. Algorithm Choice: Select algorithms appropriate for the type of heterogeneity present and the specific research question (e.g., ICP for identifying parents of $Y$, broader methods for full graph discovery).
4. Software: Libraries like causal-learn in Python offer implementations of some algorithms capable of handling heterogeneous data or incorporating environmental information. Specialized packages for methods like ICP might also be available directly from researchers' repositories.

Integrating data from heterogeneous sources provides a powerful avenue for more reliable causal discovery. By treating environmental variation not as noise but as a signal, we can impose stronger constraints on the possible causal structures, leading to more accurate inferences about the underlying data-generating processes.