Merging and Disentangling Views in Visual Reinforcement Learning for Robotic Manipulation

2025/05/07 by Almuzairee, Abdulaziz, Patil, Rohan, Bhatt, Dwait +1 · 1 citation
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotics (cs.RO)

paper · doi:10.48550/arxiv.2505.04619

Abstract

Vision is well-known for its use in manipulation, especially using visual servoing. Due to the 3D nature of the world, using multiple camera views and merging them creates better representations for Q-learning and in turn, trains more sample efficient policies. Nevertheless, these multi-view policies are sensitive to failing cameras and can be burdensome to deploy. To mitigate these issues, we introduce a Merge And Disentanglement (MAD) algorithm that efficiently merges views to increase sample efficiency while simultaneously disentangling views by augmenting multi-view feature inputs with single-view features. This produces robust policies and allows lightweight deployment. We demonstrate the efficiency and robustness of our approach using Meta-World and ManiSkill3. For project website and code, see https://aalmuzairee.github.io/mad

Citations

• Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids
• Towards General-Purpose Model-Free Reinforcement Learning
• ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI
• Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learning
• Green Screen Augmentation Enables Scene Generalisation in Robotic Manipulation
• 3D-MVP: 3D Multiview Pretraining for Robotic Manipulation
• RVT-2: Learning Precise Manipulation from Few Demonstrations
• A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning
• Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesis
• Privileged Sensing Scaffolds Reinforcement Learning
• Multi-view Disentanglement for Reinforcement Learning with Multiple Cameras
• Generalizable Visual Reinforcement Learning with Segment Anything Model
• MaDi: Learning to Mask Distractions for Generalization in Visual Deep Reinforcement Learning
• Imitation Bootstrapped Reinforcement Learning
• DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization
• TD-MPC2: Scalable, Robust World Models for Continuous Control
• METRA: Scalable Unsupervised RL with Metric-Aware Abstraction
• MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation
• Champion-level drone racing using deep reinforcement learning
• MoVie: Visual Model-Based Policy Adaptation for View Generalization
• DINOv2: Learning Robust Visual Features without Supervision
• Multi-View Masked World Models for Visual Robotic Manipulation
• X-Align++: cross-modal cross-view alignment for Bird's-eye-view segmentation
• Masked World Models for Visual Control
• Vision-Based Manipulators Need to Also See from Their Hands
• Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learning
• Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulation
• Masked Autoencoders Are Scalable Vision Learners
• How to Sense the World: Leveraging Hierarchy in Multimodal Perception for Robust Reinforcement Learning Agents
• Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning
• Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentation
• Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation
• Habitat 2.0: Training Home Assistants to Rearrange their Habitat
• SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies
• Unsupervised Visual Attention and Invariance for Reinforcement Learning
• Generalization in Reinforcement Learning by Soft Data Augmentation
• Improving Generalization in Reinforcement Learning with Mixture Regularization
• Decoupling Representation Learning from Reinforcement Learning
• Automatic Data Augmentation for Generalization in Deep Reinforcement Learning
• Reinforcement Learning with Augmented Data
• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• Supervised Contrastive Learning
• CURL: Contrastive Unsupervised Representations for Reinforcement Learning
• Learning Precise 3D Manipulation from Multiple Uncalibrated Cameras
• A Simple Framework for Contrastive Learning of Visual Representations
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• Multi-View Reinforcement Learning
• Contrastive Multiview Coding
• Contrastive Multiview Coding
• Representation Learning with Contrastive Predictive Coding
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
• Sample-Efficient Deep Reinforcement Learning via Episodic Backward\n Update
• Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination
• MVSNet: Depth Inference for Unstructured Multi-view Stereo
• Diversity is All You Need: Learning Skills without a Reward Function
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
• Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700\n Robot Hours
• Continuous control with deep reinforcement learning
• Multi-view Convolutional Neural Networks for 3D Shape Recognition
• End-to-End Training of Deep Visuomotor Policies
• Playing Atari with Deep Reinforcement Learning
• Planning and acting in partially observable stochastic domains

Cited by

• ManiVID-3D: Generalizable View-Invariant Reinforcement Learning for Robotic Manipulation via Disentangled 3D Representations

Related

• Multi-view Disentanglement for Reinforcement Learning with Multiple Cameras
• ManiVID-3D: Generalizable View-Invariant Reinforcement Learning for Robotic Manipulation via Disentangled 3D Representations
• Multi-View Masked World Models for Visual Robotic Manipulation
• Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulation
• DEAR: Disentangled Environment and Agent Representations for Reinforcement Learning without Reconstruction