Advanced Causal Inference in Machine Learning Systems
Chapter 1: Structural Causal Models and Identification Strategies
Structural Causal Models Revisited
Advanced Graphical Representations: DAGs
Do-calculus Rules and Applications
Identification Beyond Standard Criteria
Addressing Cycles and Feedback in Causal Graphs
Sensitivity Analysis for Identification Assumptions
Practice: Applying Identification Logic
Chapter 2: Advanced Causal Discovery Techniques
Constraint-based Discovery Algorithms: PC, FCI Extensions
Score-based Discovery Algorithms: GES, LiNGAM
Discovery Using Interventional Data
Causal Discovery from Heterogeneous Data
Assessing Causal Discovery Algorithm Performance
Managing Latent Variables in Discovery
Hands-on Practical: Implementing Discovery Algorithms
Chapter 3: Estimating Causal Effects in High Dimensions
Double Machine Learning for Average Treatment Effects
Causal Forests for Heterogeneous Effects
Meta-Learners for CATE Estimation
Deep Learning Approaches for Effect Estimation
Techniques for High-Dimensional Confounders
Validation and Calibration of CATE Estimators
Practice: DML and Causal Forest Implementation
Chapter 4: Confronting Unobserved Confounding and Selection Bias
Advanced Instrumental Variables (IV) Methods
Deep Learning and Kernel Methods for IV
Regression Discontinuity Designs (RDD)
Difference-in-Differences (DiD) with Panel Data
Adapting Selection Bias Correction Methods
Proximal Causal Inference Principles
Hands-on Practical: IV and RDD Analysis
Chapter 5: Causal Inference for Temporal Data and Dynamic Systems
Challenges in Causal Inference with Time Series
Granger Causality: Formulation and Shortcomings
Structural Vector Autoregression (SVAR)
Dynamic Treatment Regimes and Estimation
Time Series Causal Discovery Methods
Causal Reinforcement Learning and Off-Policy Evaluation
Practice: Temporal Causal Analysis
Chapter 6: Operationalizing Causal Inference in Machine Learning Pipelines
Causal Principles in Feature Engineering and Selection
Causality-Aware Model Development
Evaluating ML Models Using Causal Concepts
Enhancing A/B Testing with Causal Inference
Monitoring ML Systems for Causal Stability
Designing Causal Inference Components for MLOps
Hands-on Practical: Building a Causally-Informed Pipeline
Causal Reinforcement Learning and Off-Policy Evaluation
Was this section helpful?
- Doubly Robust Off-Policy Value Estimation for Reinforcement Learning, Nan Jiang, Lihong Li, 2016 International Conference on Machine Learning (ICML) DOI: 10.5555/3045390.3045470 - This is a seminal paper introducing and analyzing doubly robust estimators specifically for off-policy evaluation in reinforcement learning, addressing the high variance of importance sampling.
- Causal Inference in Sequential Decision-Making: The Confounding-Bridge Approach, Elias Bareinboim, Andrew Forney, Judea Pearl, 2015 AAAI Conference on Artificial Intelligence, Vol. 29 (Association for the Advancement of Artificial Intelligence (AAAI)) DOI: 10.1609/aaai.v29i1.9344 - This paper establishes a theoretical foundation for causal inference in dynamic systems and sequential decision-making, which is essential for understanding the causal perspective on RL and OPE.
- Proximal Reinforcement Learning: A Causal Framework for Hidden Confounding in RL, Yue Shi, Eric Tchetgen Tchetgen, Ioana Bica, 2022 Advances in Neural Information Processing Systems (NeurIPS), Vol. 35 DOI: 10.48550/arXiv.2206.01429 - This paper introduces proximal causal inference methods to address the problem of unobserved confounding in off-policy evaluation and learning within reinforcement learning settings.