Unlocking Slot Attention by Changing Optimal Transport Costs

2023/01/30 by Yan Zhang, Zhang, Yan, David W. Zhang +7 · 3 citations
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Visual Attention and Saliency Detection

paper · doi:10.48550/arxiv.2301.13197

openalex publication_date 2023/01/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

Slot attention is a powerful method for object-centric modeling in images and videos. However, its set-equivariance limits its ability to handle videos with a dynamic number of objects because it cannot break ties. To overcome this limitation, we first establish a connection between slot attention and optimal transport. Based on this new perspective we propose MESH (Minimize Entropy of Sinkhorn): a cross-attention module that combines the tiebreaking properties of unregularized optimal transport with the speed of regularized optimal transport. We evaluate slot attention using MESH on multiple object-centric learning benchmarks and find significant improvements over slot attention in every setting.

Cited by

• PEEK: Guiding and Minimal Image Representations for Zero-Shot Generalization of Robot Manipulation Policies
• Neurosymbolic Object-Centric Learning with Distant Supervision
• Unsupervised Evolutionary Cell Type Matching via Entropy-Minimized Optimal Transport

Related

• Attention Normalization Impacts Cardinality Generalization in Slot Attention
• Adaptive Slot Attention: Object Discovery with Dynamic Slot Number
• Improving Object-centric Learning with Query Optimization
• Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learning
• Slot-Guided Adaptation of Pre-trained Diffusion Models for Object-Centric Learning and Compositional Generation