Challenges in Grounding Language in the Real World

2025/06/20 by Lindes, Peter, Skiker, Kaoutar
#Artificial Intelligence (cs.AI) #FOS: Biological sciences #FOS: Computer and information sciences #Neurons and Cognition (q-bio.NC)

paper · doi:10.48550/arxiv.2506.17375

Abstract

A long-term goal of Artificial Intelligence is to build a language understanding system that allows a human to collaborate with a physical robot using language that is natural to the human. In this paper we highlight some of the challenges in doing this, and propose a solution that integrates the abilities of a cognitive agent capable of interactive task learning in a physical robot with the linguistic abilities of a large language model. We also point the way to an initial implementation of this approach.

Related

• Using Natural Language for Human-Robot Collaboration in the Real World
• Natural Language for Human-Robot Collaboration: Problems Beyond Language Grounding
• Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
• A Survey of Language-Based Communication in Robotics
• Grounded Language Learning in a Simulated 3D World