Bridging Perception and Planning: Towards End-to-End Planning for Signal Temporal Logic Tasks

2025/09/16 by Bowen Ye, Ye, Bowen, Huang, Junyue +5 · 1 citation
Computer Science · #Logic, Reasoning, and Knowledge #Semantic Web and Ontologies #AI-based Problem Solving and Planning

paper · pdf · doi:10.48550/arxiv.2509.12813

Abstract

We investigate the task and motion planning problem for Signal Temporal Logic (STL) specifications in robotics. Existing STL methods rely on pre-defined maps or mobility representations, which are ineffective in unstructured real-world environments. We propose the Structured-MoE STL Planner (S-MSP), a differentiable framework that maps synchronized multi-view camera observations and an STL specification directly to a feasible trajectory. S-MSP integrates STL constraints within a unified pipeline, trained with a composite loss that combines trajectory reconstruction and STL robustness. A structure-aware Mixture-of-Experts (MoE) model enables horizon-aware specialization by projecting sub-tasks into temporally anchored embeddings. We evaluate S-MSP using a high-fidelity simulation of factory-logistics scenarios with temporally constrained tasks. Experiments show that S-MSP outperforms single-expert baselines in STL satisfaction and trajectory feasibility. A rule-based safety filter at inference improves physical executability without compromising logical correctness, showcasing the practicality of the approach.

Citations

• DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving
• Zero-Shot Trajectory Planning for Signal Temporal Logic Tasks
• Symbolic Control for Autonomous Docking of Marine Surface Vessels
• DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving
• π₀: A Vision-Language-Action Flow Model for General Robot Control
• MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learning
• OpenVLA: An Open-Source Vision-Language-Action Model
• Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning
• Formal Synthesis of Controllers for Safety-Critical Autonomous Systems: Developments and Challenges
• DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
• Signal Temporal Logic Control Synthesis among Uncontrollable Dynamic Agents with Conformal Prediction
• SARA-RT: Scaling up Robotics Transformers with Self-Adaptive Robust Attention
• Sleep When Everything Looks Fine: Self-Triggered Monitoring for Signal Temporal Logic Tasks
• Signal Temporal Logic Neural Predictive Control
• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
• VAD: Vectorized Scene Representation for Efficient Autonomous Driving
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
• Diffusion policy: Visuomotor policy learning via action diffusion
• Planning-oriented Autonomous Driving
• RT-1: Robotics Transformer for Real-World Control at Scale
• Backpropagation through Signal Temporal Logic Specifications: Infusing Logical Structure into Gradient-Based Methods
• A Smooth Robustness Measure of Signal Temporal Logic for Symbolic Control
• TransDiffuser: Diverse Trajectory Generation with Decorrelated Multi-modal Representation for End-to-end Autonomous Driving

Cited by

• Vision-TL-Action: Neuro-Symbolic Trajectory Generation from Visual Observations and Temporal Logic

Related

• Zero-Shot Trajectory Planning for Signal Temporal Logic Tasks
• Signal Temporal Logic Compliant Co-design of Planning and Control
• Time-Robust Path Planning with Piece-Wise Linear Trajectory for Signal Temporal Logic Specifications
• T3 Planner: A Self-Correcting LLM Framework for Robotic Motion Planning with Temporal Logic
• Multi-agent Motion Planning from Signal Temporal Logic Specifications