Sim and Real: Better Together

2021/10/01 by Shirli Di Castro Shashua, Shashua, Shirli Di Castro, Dotan Di Castro +3
Computer Science · Decision Sciences · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Parallel Computing and Optimization Techniques #Scientific Computing and Data Management #Simulation Techniques and Applications

paper · pdf · doi:10.48550/arxiv.2110.00445

openalex publication_date 2021/10/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Simulation is used extensively in autonomous systems, particularly in robotic manipulation. By far, the most common approach is to train a controller in simulation, and then use it as an initial starting point for the real system. We demonstrate how to learn simultaneously from both simulation and interaction with the real environment. We propose an algorithm for balancing the large number of samples from the high throughput but less accurate simulation and the low-throughput, high-fidelity and costly samples from the real environment. We achieve that by maintaining a replay buffer for each environment the agent interacts with. We analyze such multi-environment interaction theoretically, and provide convergence properties, through a novel theoretical replay buffer analysis. We demonstrate the efficacy of our method on a sim-to-real environment.

Related

• Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identifications
• Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twin
• What Went Wrong? Closing the Sim-to-Real Gap via Differentiable Causal Discovery
• Closing the Sim-to-Real Loop: Adapting Simulation Randomization with\n Real World Experience
• Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL