ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation

2025/05/28 by Jiawen Yu, Yu, Jiawen, H X Liu +19 · 19 citations
Engineering · Neuroscience · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Robot Manipulation and Learning #Robotics (cs.RO) #Tactile and Sensory Interactions #Teleoperation and Haptic Systems

paper · pdf · doi:10.48550/arxiv.2505.22159

openalex publication_date 2025/05/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Vision-Language-Action (VLA) models have advanced general-purpose robotic manipulation by leveraging pretrained visual and linguistic representations. However, they struggle with contact-rich tasks that require fine-grained control involving force, especially under visual occlusion or dynamic uncertainty. To address these limitations, we propose ForceVLA, a novel end-to-end manipulation framework that treats external force sensing as a first-class modality within VLA systems. ForceVLA introduces FVLMoE, a force-aware Mixture-of-Experts fusion module that dynamically integrates pretrained visual-language embeddings with real-time 6-axis force feedback during action decoding. This enables context-aware routing across modality-specific experts, enhancing the robot's ability to adapt to subtle contact dynamics. We also introduce ForceVLA-Data, a new dataset comprising synchronized vision, proprioception, and force-torque signals across five contact-rich manipulation tasks. ForceVLA improves average task success by 23.2% over strong pi₀-based baselines, achieving up to 80% success in tasks such as plug insertion. Our approach highlights the importance of multimodal integration for dexterous manipulation and sets a new benchmark for physically intelligent robotic control. Code and data will be released at https://sites.google.com/view/forcevla2025.

Cited by

• τ: Learning Touch-Augmented Vision-Language-Action Models from Future Visual Supervision
• Embodied Robot Manipulation in the Era of Foundation Models: Planning and Learning Perspectives
• N₀-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulation
• N₀-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens
• Sample-Efficient Robot Skill Learning for Construction Tasks: Benchmarking Hierarchical Reinforcement Learning and Vision-Language-Action VLA Model
• An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges
• Data-driven Interpretable Hybrid Robot Dynamics
• WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Driving
• VacuumVLA: Boosting VLA Capabilities via a Unified Suction and Gripping Tool for Complex Robotic Manipulation
• When Alignment Fails: Multimodal Adversarial Attacks on Vision-Language-Action Models
• Explicit Kinematic Guidance from Analytic Concepts for Vision-Language-Action Models
• Expertise need not monopolize: Action-Specialized Mixture of Experts for Vision-Language-Action Learning
• Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
• dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought
• Pure Vision Language Action (VLA) Models: A Comprehensive Survey
• Survey of Vision-Language-Action Models for Embodied Manipulation
• Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey
• Human Centric General Physical Intelligence for Agile Manufacturing Automation
• OmniVTLA: Vision-Tactile-Language-Action Model with Semantic-Aligned Tactile Sensing

Related

• FM-VLA: Force-based Memory for Vision-Language-Action Models in Contact-Rich Manipulation
• TA-VLA: Elucidating the Design Space of Torque-aware Vision-Language-Action Models
• Never Too Late for Force: Accelerating VLA Post-Training with Reactive Force Injection
• Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA
• VTLA: Vision-Tactile-Language-Action Model with Preference Learning for Insertion Manipulation