2025/09/17 by Weng, Zhengyang Kris, Matthew L. Elwin, Han Liu +2
Computer Science · Engineering · Medicine · #Hand Gesture Recognition Systems #Teleoperation and Haptic Systems #Stroke Rehabilitation and Recovery
paper · pdf · doi:10.48550/arxiv.2509.14349
Hand Tracking Streamer is a lightweight hand telemetry utility that turns a Meta Quest headset into a precision controller for robotics teleoperation and motion capture. Built on the Meta Interaction SDK, it streams 21-landmark hand data and 6-DoF wrist pose to a PC in real time over Wi-Fi using UDP for ultra-low latency or TCP for reliable data logging. The app supports left, right, or dual-hand modes with in-headset configuration, and includes a live log console and phantom hand visualization for debugging. Data is sent as structured packets of joint positions and orientations in Unity world space, making it suitable for robot control, imitation learning, and gesture-based prototyping. Open source available on github. Read more about connection and deployment examples on: https://github.com/wengmister/hand-tracking-streamer