Alignment with human representations supports robust few-shot learning

2023/01/27 by Ilia Sucholutsky, Thomas L. Griffiths, Sucholutsky, Ilia +1 · 3 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Bacillus and Francisella bacterial research #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2301.11990

openalex publication_date 2023/01/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Should we care whether AI systems have representations of the world that are similar to those of humans? We provide an information-theoretic analysis that suggests that there should be a U-shaped relationship between the degree of representational alignment with humans and performance on few-shot learning tasks. We confirm this prediction empirically, finding such a relationship in an analysis of the performance of 491 computer vision models. We also show that highly-aligned models are more robust to both natural adversarial attacks and domain shifts. Our results suggest that human-alignment is often a sufficient, but not necessary, condition for models to make effective use of limited data, be robust, and generalize well.

Cited by

• Uncovering the Computational Ingredients of Human-Like Representations in LLMs
• Dissecting the effectiveness of deep features as metric of perceptual image quality
• Brain2Model Transfer: Training sensory and decision models with human neural activity as a teacher

Related

• Alignment and Adversarial Robustness: Are More Human-Like Models More Secure?
• Human alignment of neural network representations
• Aligning Machine and Human Visual Representations across Abstraction Levels
• Do Machines Fail Like Humans? A Human-Centred Out-of-Distribution Spectrum for Mapping Error Alignment
• Evaluating alignment between humans and neural network representations in image-based learning tasks