2025/06/28 by Qixuan Liu, Liu, Qixuan, Shi Qiu +11
Computer Science · Engineering · #Anatomy and Medical Technology #Computer Graphics and Visualization Techniques #FOS: Computer and information sciences #Graphics (cs.GR) #Human-Computer Interaction (cs.HC) #Interactive and Immersive Displays #Multimedia (cs.MM)
paper · pdf · doi:10.48550/arxiv.2506.22926
openalex publication_date 2025/06/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Volumetric medical imaging technologies produce detailed 3D representations of anatomical structures. However, effective medical data visualization and exploration pose significant challenges, especially for individuals with limited medical expertise. We introduce a novel XR-based system with two key innovations: (1) a coordinated visualization module integrating Multi-layered Multi-planar Reconstruction with 3D mesh models and (2) a multimodal interaction framework combining hand gestures with LLM-enabled voice commands. We conduct preliminary evaluations, including a 15-participant user study and expert interviews, to demonstrate the system's abilities to enhance spatial understanding and reduce cognitive load. Experimental results show notable improvements in task completion times, usability metrics, and interaction effectiveness enhanced by LLM-driven voice control. While identifying areas for future refinement, our findings highlight the potential of this immersive visualization system to advance medical training and clinical practice. Our demo application and supplemental materials are available for download at: https://osf.io/bpjq5/.