A Causal Ordering Prior for Unsupervised Representation Learning

2023/07/11 by Kori, Avinash, Sanchez, Pedro, Vilouras, Konstantinos +2 · 1 citation
#Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2307.05704

Abstract

Unsupervised representation learning with variational inference relies heavily on independence assumptions over latent variables. Causal representation learning (CRL), however, argues that factors of variation in a dataset are, in fact, causally related. Allowing latent variables to be correlated, as a consequence of causal relationships, is more realistic and generalisable. So far, provably identifiable methods rely on: auxiliary information, weak labels, and interventional or even counterfactual data. Inspired by causal discovery with functional causal models, we propose a fully unsupervised representation learning method that considers a data generation process with a latent additive noise model (ANM). We encourage the latent space to follow a causal ordering via loss function based on the Hessian of the latent distribution.

Cited by

• Debiasing Reward Models by Representation Learning with Guarantees

Related

• Identifying Weight-Variant Latent Causal Models
• Identifying Weight-Variant Latent Causal Models
• Causal Representation Learning Made Identifiable by Grouping of Observational Variables
• Generalizable Information Theoretic Causal Representation
• Linear Causal Representation Learning by Topological Ordering, Pruning, and Disentanglement