2024/11/26 by Shipeng Liu, FNU Shrutika, Liu, Shipeng +7
Computer Science · Decision Sciences · Psychology · #68T05 #AI in Service Interactions #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Human-Automation Interaction and Safety #Human-Computer Interaction (cs.HC) #I.2.9 #Impact of AI and Big Data on Business and Society #Robotics (cs.RO)
paper · pdf · doi:10.48550/arxiv.2412.06808
openalex publication_date 2024/11/26 · openalex created_date 2024/12/12 · openalex updated_date 2026/07/28
Effective human-robot collaboration requires robot to adopt their roles and levels of support based on human needs, task requirements, and complexity. Traditional human-robot teaming often relies on a pre-determined robot communication scheme, restricting teamwork adaptability in complex tasks. Leveraging strong communication capabilities of Large Language Models (LLMs), we propose a Human-Robot Teaming Framework with Multi-Modal Language feedback (HRT-ML), a framework designed to enhance human-robot interaction by adjusting the frequency and content of language-based feedback. HRT-ML framework includes two core modules: a Coordinator for high-level, low-frequency strategic guidance, and a Manager for subtask-specific, high-frequency instructions, enabling passive and active interactions with human teammates. To assess the impact of language feedback in collaborative scenarios, we conducted experiments in an enhanced Overcooked environment with varying levels of task complexity (easy, medium, hard) and feedback frequency (inactive, passive, active, superactive). Our results show that as task complexity increases relative to human capabilities, human teammates exhibited a stronger preference towards robotic agents that can offer frequent, proactive support. However, when task complexities exceed the LLM's capacity, noisy and inaccurate feedback from superactive robotic agents can instead hinder team performance, as it requires human teammates to increase their effort to interpret and respond to a large number of communications, with limited performance return. Our results offer a general principle for robotic agents to dynamically adjust their levels and frequencies of communications to work seamlessly with humans and achieve improved teaming performance.