2021/09/07 by Haoran Xie, Yichen Peng, Xie, Haoran +5
Computer Science · #Augmented Reality Applications #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Interactive and Immersive Displays #Virtual Reality Applications and Impacts
paper · pdf · doi:10.48550/arxiv.2109.03013
openalex publication_date 2021/09/07 · openalex created_date 2021/09/13 · openalex updated_date 2026/07/28
In this work, we propose an interactive general instruction framework SketchMeHow to guidance the common users to complete the daily tasks in real-time. In contrast to the conventional augmented reality-based instruction systems, the proposed framework utilizes the user sketches as system inputs to acquire the users' production intentions from the drawing interfaces. Given the user sketches, the designated task instruction can be analyzed based on the sub-task division and spatial localization for each task. The projector-camera system is adopted in the projection guidance to the end-users with the spatial augmented reality technology. To verify the proposed framework, we conducted two case studies of domino arrangement and bento production. From our user studies, the proposed systems can help novice users complete the tasks efficiently with user satisfaction. We believe the proposed SketchMeHow can broaden the research topics in sketch-based real-world applications in human-computer interaction.