2025/01/18 by Oliver Chojnowski, Chojnowski, Oliver, Alexander Eberhard +7 · 1 citation
Computer Science · Engineering · #FOS: Computer and information sciences #Human Motion and Animation #Human Pose and Action Recognition #Human-Computer Interaction (cs.HC) #Robotics (cs.RO) #Video Analysis and Summarization
paper · pdf · doi:10.48550/arxiv.2501.10713
openalex publication_date 2025/01/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Socially interactive agents are gaining prominence in domains like healthcare, education, and service contexts, particularly virtual agents due to their inherent scalability. To facilitate authentic interactions, these systems require verbal and nonverbal communication through e.g., facial expressions and gestures. While natural language processing technologies have rapidly advanced, incorporating human-like nonverbal behavior into real-world interaction contexts is crucial for enhancing the success of communication, yet this area remains underexplored. One barrier is creating autonomous systems with sophisticated conversational abilities that integrate human-like nonverbal behavior. This paper presents a distributed architecture using Epic Games MetaHuman, combined with advanced conversational AI and camera-based user management, that supports methods like motion capture, handcrafted animation, and generative approaches for nonverbal behavior. We share insights into a system architecture designed to investigate nonverbal behavior in socially interactive agents, deployed in a three-week field study in the Deutsches Museum Bonn, showcasing its potential in realistic nonverbal behavior research.