2026/08/01 by Ivan Snegirev, Elizaveta Semenyakina, Mikhail Konenkov +3
Computer Science · #cs.RO #cs.CV #cs.HC #msc:68T40 #msc:68U35 #msc:68T50 #acm:68T40 #acm:68U35 #acm:68T50
4 page, 3 figures, 1 table, ISMAR 2026
arxiv created 2026/08/01 · arxiv updated 2026/08/04
ORCESTRA is a mixed-reality system for programming robot digital twins through no-code waypoint teaching and language-guided control. In a passthrough mixed-reality workspace, users place robot twins on real surfaces, teach trajectories, save robot-relative episodes, or issue spoken/typed commands that a vision-language model converts into structured digital-twin plans. Both interaction modes share a backend for metric grounding, embodiment-aware validation, preview, confirmation, and digital-twin execution. The system supports heterogeneous robot embodiments, including fixed-base manipulators, a mobile base, and a humanoid robot, demonstrating MR validation as a safety layer for language-guided robot programming before physical deployment.