From Training to Deployment: Post-Hoc Causal Feature Identification via Sensitivity Ratios

2026/07/28 by Athanasios Vlontzos, Giorgos Papanastasiou, Bernhard Kainz +1
#cs.AI

paper · pdf

Abstract

Given a model that is already trained, which features does it rely on causally versus spuriously? Existing methods require access to the training procedure and cannot answer this post-hoc. We introduce the Normalised Sensitivity Ratio~(NSR), a post-hoc, model-agnostic diagnostic for this question under a structured-shift regime: environments differ primarily in the mean of spurious features while the causal mechanism and causal marginals remain stable, as in multi-site clinical data or multi-batch genomics. Within this regime, causal features induce constant model sensitivity across environments while spurious features track shift. NSR formalises this as the squared coefficient of variation of per-environment sensitivity. Under a linear structural causal model (SCM) with K≥3 non-degenerate environments, NSR achieves exact identification (Theorem~1). We fully characterise failure: weak shifts (O(ε⁴) collapse), degenerate geometry, and proxy attenuation (O((1-α)⁴)), giving practitioners quantitative criteria for assessing whether the regime holds. Finite-sample rates are O_p(n^-1) under the null and O_p(n^-1/2) under the alternative. Experiments confirm all theoretical predictions on synthetic data (area under the ROC curve [AUROC] = 1.000 under conditions satisfying the regime), show consistent rankings across five model families (Kendall τ≥0.529), and recover six of eight causal features on bike-sharing data (Precision@7 = 0.75) without modifying any trained model.

Citations

• Contrastive representations of high-dimensional, structured treatments
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• Invariant Risk Minimization
• Causal inference using invariant prediction: identification and confidence intervals
• Modeling wine preferences by data mining from physicochemical properties

Related

• Counterfactual Normalization: Proactively Addressing Dataset Shift and Improving Reliability Using Causal Mechanisms
• Tracking the risk of a deployed model and detecting harmful distribution shifts
• Considerations for Distribution Shift Robustness of Diagnostic Models in Healthcare
• Preventing Failures Due to Dataset Shift: Learning Predictive Models That Transport
• Stress-Testing Assumptions: A Guide to Bayesian Sensitivity Analyses in Causal Inference