Path Integral Networks: End-to-End Differentiable Optimal Control

2017/06/29 by Masashi Okada, Okada, Masashi, Luca Rigazio +3 · 41 citations
Computer Science · Engineering · Mathematics · #Advanced Control Systems Optimization #Artificial Intelligence (cs.AI) #Computer network #Computer science #Differentiable function #End-to-end principle #FOS: Computer and information sciences #FOS: Electrical engineering #Mathematical analysis #Mathematics #Neural Networks Stability and Synchronization #Path (computing) #Reinforcement Learning in Robotics #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.1706.09597

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2017/06/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

In this paper, we introduce Path Integral Networks (PI-Net), a recurrent network representation of the Path Integral optimal control algorithm. The network includes both system dynamics and cost models, used for optimal control based planning. PI-Net is fully differentiable, learning both dynamics and cost models end-to-end by back-propagation and stochastic gradient descent. Because of this, PI-Net can learn to plan. PI-Net has several advantages: it can generalize to unseen states thanks to planning, it can be applied to continuous control tasks, and it allows for a wide variety learning schemes, including imitation and reinforcement learning. Preliminary experiment results show that PI-Net, trained by imitation learning, can mimic control demonstrations for two simulated problems; a linear system and a pendulum swing-up problem. We also show that PI-Net is able to learn dynamics and cost models latent in the demonstrations.

Citations

• Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
• End to End Learning for Self-Driving Cars
• DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
• A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

Cited by

• QMDP-Net: Deep Learning for Planning under Partial Observability
• PlaNet of the Bayesians: Reconsidering and Improving Deep Planning Network by Incorporating Bayesian Inference
• On the model-based stochastic value gradient for continuous reinforcement learning
• Learning Convex Optimization Control Policies
• DiffCoord: Differentiable Coordination for Distributed Multi-Agent Trajectory Optimization
• Differentiable MPC for End-to-end Planning and Control
• Distilling a Hierarchical Policy for Planning and Control via Representation and Reinforcement Learning
• Learning Navigation Costs from Demonstration in Partially Observable Environments
• Particle Filter Networks with Application to Visual Localization
• DiLQR: Differentiable Iterative Linear Quadratic Regulator via Implicit Differentiation
• Disentangling Controllable and Uncontrollable Factors of Variation by Interacting with the World
• The Differentiable Cross-Entropy Method

Related

• Sample Efficient Path Integral Control under Uncertainty
• GPU Based Path Integral Control with Learned Dynamics
• Adaptive Smoothing Path Integral Control
• Recent Advances in Path Integral Control for Trajectory Optimization: An Overview in Theoretical and Algorithmic Perspectives
• Coordinating Planning and Tracking in Layered Control Policies via Actor-Critic Learning