2023/08/04 by Michele Yin, Gabriel Roccabruna, Yin, Michele +5
Computer Science · #AI in Service Interactions #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Speech and dialogue systems
paper · pdf · doi:10.48550/arxiv.2308.02665
openalex publication_date 2023/08/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The dialogue experience with conversational agents can be greatly enhanced with multimodal and immersive interactions in virtual reality. In this work, we present an open-source architecture with the goal of simplifying the development of conversational agents operating in virtual environments. The architecture offers the possibility of plugging in conversational agents of different domains and adding custom or cloud-based Speech-To-Text and Text-To-Speech models to make the interaction voice-based. Using this architecture, we present two conversational prototypes operating in the digital health domain developed in Unity for both non-immersive displays and VR headsets.