2024/12/30 by Mykola Maslych, Maslych, Mykola, Christian Pumarada +5 · 6 citations
Computer Science · #AI in Service Interactions #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC)
paper · pdf · doi:10.48550/arxiv.2501.00168
openalex publication_date 2024/12/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We present a virtual reality (VR) environment featuring conversational avatars powered by a locally-deployed LLM, integrated with automatic speech recognition (ASR), text-to-speech (TTS), and lip-syncing. Through a pilot study, we explored the effects of three types of avatar status indicators during response generation. Our findings reveal design considerations for improving responsiveness and realism in LLM-driven conversational systems. We also detail two system architectures: one using an LLM-based state machine to control avatar behavior and another integrating retrieval-augmented generation (RAG) for context-grounded responses. Together, these contributions offer practical insights to guide future work in developing task-oriented conversational AI in VR environments.