Context-aware Learned Mesh-based Simulation via Trajectory-Level Meta-Learning
2025/11/07 by Dahlinger, Philipp, Freymuth, Niklas, Hoang, Tai +4
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotics (cs.RO)
paper · doi:10.48550/arxiv.2511.05234
Abstract
Simulating object deformations is a critical challenge across many scientific domains, including robotics, manufacturing, and structural mechanics. Learned Graph Network Simulators (GNSs) offer a promising alternative to traditional mesh-based physics simulators. Their speed and inherent differentiability make them particularly well suited for applications that require fast and accurate simulations, such as robotic manipulation or manufacturing optimization. However, existing learned simulators typically rely on single-step observations, which limits their ability to exploit temporal context. Without this information, these models fail to infer, e.g., material properties. Further, they rely on auto-regressive rollouts, which quickly accumulate error for long trajectories. We instead frame mesh-based simulation as a trajectory-level meta-learning problem. Using Conditional Neural Processes, our method enables rapid adaptation to new simulation scenarios from limited initial data while capturing their latent simulation properties. We utilize movement primitives to directly predict fast, stable and accurate simulations from a single model call. The resulting approach, Movement-primitive Meta-MeshGraphNet (M3GN), provides higher simulation accuracy at a fraction of the runtime cost compared to state-of-the-art GNSs across several tasks.
Citations
- PAINT: Parallel-in-time Neural Twins for Dynamical System Reconstruction
- MaNGO - Adaptable Graph Network Simulators via Meta-Learning
- Diffusion-Based Hierarchical Graph Neural Networks for Simulating Nonlinear Solid Mechanics
- AMBER: Adaptive Mesh Generation by Iterative Mesh Resolution Prediction
- Geometry-aware RL for Manipulation of Varying Shapes and Deformable Objects
- DEL: Discrete Element Learner for Learning 3D Particle Dynamics with Neural Rendering
- Scaling Face Interaction Graph Networks to Real World Scenes
- Open the Black Box: Step-based Policy Updates for Temporally-Correlated Episodic Reinforcement Learning
- Equivariant Graph Neural Operator for Modeling 3D Dynamics
- Learning Flexible Body Collision Dynamics with Hierarchical Contact Mesh Transformer
- Learning 3D Particle-based Simulators from RGB-D Videos
- Graph Deep Learning for Time Series Forecasting
- PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
- RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools
- Learning Controllable Adaptive Simulation for Multi-resolution Physics
- Swarm Reinforcement Learning For Adaptive Mesh Refinement
- Grounding Graph Network Simulators using Physical Sensor Observations
- Learning rigid dynamics with face interaction graph networks
- Deep Black-Box Reinforcement Learning with Movement Primitives
- ProDMPs: A Unified Perspective on Dynamic and Probabilistic Movement Primitives
- Message Passing Neural PDE Solvers
- Predicting Physics in Mesh-reduced Space with Temporal Attention
- A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulation
- DEBOSH: Deep Bayesian Shape Optimization
- Deep Reinforcement Learning at the Edge of the Statistical Precipice
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
- Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects
- ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations
- Neural Dynamic Policies for End-to-End Sensorimotor Learning
- Meta-Learning in Neural Networks: A Survey
- Learning to Simulate Complex Physics with Graph Networks
- A Comprehensive Survey on Transfer Learning
- A Comprehensive Survey on Transfer Learning
- The Functional Neural Process
- Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization
- Attentive Neural Processes
- Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids
- Neural Processes
- Conditional Neural Processes
- Bayesian Model-Agnostic Meta-Learning
- Probabilistic Model-Agnostic Meta-Learning
- Graph networks as learnable physics engines for inference and control
- Relational inductive biases, deep learning, and graph networks
- Geodesic Convolutional Shape Optimization
- Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
- Prototypical Networks for Few-shot Learning
- Deep Sets
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Interaction Networks for Learning about Objects, Relations and Physics
- Learning to learn by gradient descent by gradient descent
- ImageNet classification with deep convolutional neural networks
- Isogeometric analysis: CAD, finite elements, NURBS, exact geometry and mesh refinement
- Long Short-Term Memory
- A Foundation Model for the Earth System
Related