2025/06/01 by David Sasu, Sasu, David, Yamoah, Kweku Andoh +4
Computer Science · Engineering · Psychology · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotics (cs.RO) #Robotics and Automated Systems #Social Robot Interaction and HRI #Speech and dialogue systems
paper · pdf · doi:10.48550/arxiv.2506.02057
openalex publication_date 2025/06/01 · openalex created_date 2025/10/14 · openalex updated_date 2026/07/28
Enabling robots to accurately interpret and execute spoken language instructions is essential for effective human-robot collaboration. Traditional methods rely on speech recognition to transcribe speech into text, often discarding crucial prosodic cues needed for disambiguating intent. We propose a novel approach that directly leverages speech prosody to infer and resolve instruction intent. Predicted intents are integrated into large language models via in-context learning to disambiguate and select appropriate task plans. Additionally, we present the first ambiguous speech dataset for robotics, designed to advance research in speech disambiguation. Our method achieves 95.79% accuracy in detecting referent intents within an utterance and determines the intended task plan of ambiguous instructions with 71.96% accuracy, demonstrating its potential to significantly improve human-robot communication.