2021/07/19 by Petar Veličković, Matko Bošnjak, Veličković, Petar +11 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2107.08881
openalex publication_date 2021/07/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Neural networks leverage robust internal representations in order to generalise. Learning them is difficult, and often requires a large training set that covers the data distribution densely. We study a common setting where our task is not purely opaque. Indeed, very often we may have access to information about the underlying system (e.g. that observations must obey certain laws of physics) that any "tabula rasa" neural network would need to re-learn from scratch, penalising performance. We incorporate this information into a pre-trained reasoning module, and investigate its role in shaping the discovered representations in diverse self-supervised learning settings from pixels. Our approach paves the way for a new class of representation learning, grounded in algorithmic priors.