2017/05/29 by Katrina Evtimova, Andrew Drozdov, Evtimova, Katrina +5 · 3 citations
Computer Science · Social Sciences · #Artificial Intelligence in Games #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Evolutionary Game Theory and Cooperation #FOS: Computer and information sciences #Information Theory (cs.IT) #Language and cultural evolution #Machine Learning (cs.LG) #Multiagent Systems (cs.MA)
paper · pdf · doi:10.48550/arxiv.1705.10369
openalex publication_date 2017/05/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Inspired by previous work on emergent communication in referential games, we propose a novel multi-modal, multi-step referential game, where the sender and receiver have access to distinct modalities of an object, and their information exchange is bidirectional and of arbitrary duration. The multi-modal multi-step setting allows agents to develop an internal communication significantly closer to natural language, in that they share a single set of messages, and that the length of the conversation may vary according to the difficulty of the task. We examine these properties empirically using a dataset consisting of images and textual descriptions of mammals, where the agents are tasked with identifying the correct object. Our experiments indicate that a robust and efficient communication protocol emerges, where gradual information exchange informs better predictions and higher communication bandwidth improves generalization.