An Introduction to Zero-Order Optimization Techniques for Robotics

2025/06/27 by Jordana, Armand, Zhang, Jianghan, Amigo, Joseph +1 · 2 citations
#FOS: Computer and information sciences #Robotics (cs.RO)

paper · doi:10.48550/arxiv.2506.22087

Abstract

Zero-order optimization techniques are becoming increasingly popular in robotics due to their ability to handle non-differentiable functions and escape local minima. These advantages make them particularly useful for trajectory optimization and policy optimization. In this work, we propose a mathematical tutorial on random search. It offers a simple and unifying perspective for understanding a wide range of algorithms commonly used in robotics. Leveraging this viewpoint, we classify many trajectory optimization methods under a common framework and derive novel competitive RL algorithms.

Citations

• Stein Variational Evolution Strategies
• Full-Order Sampling-Based MPC for Torque-Level Locomotion Control via Diffusion-Style Annealing
• DROP: Dexterous Reorientation via Online Planning
• NLP Sampling: Combining MCMC and NLP Methods for Diverse Constrained Sampling
• ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots
• evosax: JAX-based Evolution Strategies
• Predictive Sampling: Real-time Behaviour Synthesis with MuJoCo
• DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality
• Combining Evolution and Deep Reinforcement Learning for Policy Search: a Survey
• Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems
• Bundled Gradients through Contact via Randomized Smoothing
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Is the Policy Gradient a Gradient?
• A Theoretical and Empirical Comparison of Gradient Approximations in\n Derivative-Free Optimization
• An Online Learning Approach to Model Predictive Control
• Policy Search in Continuous Action Domains: an Overview
• Addressing Function Approximation Error in Actor-Critic Methods
• Stochastic Gradient Descent as Approximate Bayesian Inference
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• Stein Variational Gradient Descent: A General Purpose Bayesian Inference\n Algorithm
• Continuous control with deep reinforcement learning
• The Metropolis-Hastings algorithm
• Coordinate Descent Algorithms
• Path Integral Policy Improvement with Covariance Matrix Adaptation
• Online convex optimization in the bandit setting: gradient descent without a gradient
• Natural Gradient Works Efficiently in Learning
• Recursive Stochastic Algorithms for Global Optimization in ℝ^d
• Reinforcement Learning: An Introduction

Cited by

• Reference-Free Sampling-Based Model Predictive Control
• WorldPlanner: Monte Carlo Tree Search and MPC with Action-Conditioned Visual World Models

Related

• Learning the Problem-Optimum Map: Analysis and Application to Global\n Optimization in Robotics
• iDb-RRT: Sampling-based Kinodynamic Motion Planning with Motion Primitives and Trajectory Optimization
• Sampling-based Algorithms for Optimal Motion Planning Using Closed-loop\n Prediction
• FATROP : A Fast Constrained Optimal Control Problem Solver for Robot Trajectory Optimization and Control
• Combining Benefits from Trajectory Optimization and Deep Reinforcement Learning