Advances in Causal Representation Learning: From i.i.d. to Non‑Stationary Settings

01 Why Causal Relationships Matter

The article begins by defining causal relations as interventions that change a target variable and explains why understanding causality is crucial, using three classic examples: the link between smoking, lung disease and nail color; Simpson's paradox in treatment effectiveness; and selection bias in gender‑related statistics.

These examples illustrate how causal analysis can reveal underlying mechanisms that simple correlations cannot.

02 Causal Representation Learning under i.i.d. Assumption

The section introduces the basic concepts of causal discovery and causal representation learning, emphasizing the modularity of causal systems and the three key properties: conditional independence, independent noise, and minimal change. It discusses identifiability issues and presents two fundamental algorithms for i.i.d. data: the PC (Peter‑Clark) algorithm (assuming no hidden confounders) and the FCI algorithm (allowing latent variables).

Examples using archaeological skull measurements demonstrate how conditional independence tests can recover a directed acyclic graph (DAG) that reflects the true causal structure.

The discussion then covers three model families—Linear Non‑Gaussian Models, Post‑Nonlinear (PNL) Models, and Additive Noise Models—showing how asymmetries in residual independence can identify causal direction.

03 Causal Representation Learning from Time Series

When data are not i.i.d. but temporally ordered, the article introduces Granger causality and instantaneous causal relations. It explains how latent processes can be recovered from video or sensor streams using smooth, invertible nonlinear mappings, allowing the discovery of hidden objects and their interactions.

Illustrations include a KittiMask video where three latent motions (horizontal, vertical, mask size) are uncovered, and a synthetic multi‑ball system where latent positions and spring‑like connections are inferred.

04 Causal Representation Learning under Distribution Shifts

The final part addresses non‑stationary or heterogeneous data where the underlying distribution changes over time or across domains. It shows how causal modules can vary independently, enabling detection of which causal mechanisms have changed, reconstruction of the skeleton, and orientation of edges using the independence of changes between cause and effect.

Applications include stock‑return data from the NYSE, where sector‑wise clusters and causal dynamics across the 2007‑2008 financial crisis are visualized, and domain‑adaptation scenarios where augmented graphs capture how conditional distributions of target variables evolve across domains.

The article concludes that causal representation learning provides a powerful framework for many machine‑learning problems—decision making, domain adaptation, reinforcement learning, recommendation systems, trustworthy AI, and fairness—by enabling the recovery of true underlying causal structures from data.

Key Takeaways

A series of machine‑learning tasks require proper causal representations of data.

Under suitable assumptions, causal relations and latent variables can be fully recovered from observational data.

Causality is not mysterious; with data and verifiable assumptions, it can be identified and leveraged.