Are Transformers Truly Foundational for Robotics?

2024/11/25 by James A. R. Marshall, Andrew B. Barron, Marshall, James A. R. +1 · 2 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Reinforcement Learning in Robotics #Robot Manipulation and Learning #Robotics (cs.RO)

paper · doi:10.48550/arxiv.2411.16917

openalex publication_date 2024/11/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Generative Pre-Trained Transformers (GPTs) are hyped to revolutionize robotics. Here we question their utility. GPTs for autonomous robotics demand enormous and costly compute, excessive training times and (often) offboard wireless control. We contrast GPT state of the art with how tiny insect brains have achieved robust autonomy with none of these constraints. We highlight lessons that can be learned from biology to enhance the utility of GPTs in robotics.

Cited by

• Explainable Deep Convolutional Multi-Type Anomaly Detection
• A New Strategy for Artificial Intelligence: Training Foundation Models Directly on Human Brain Data

Related

• Advances in Transformers for Robotic Applications: A Review
• What comes after transformers? -- A selective survey connecting ideas in deep learning
• MetaMorph: Learning Universal Controllers with Transformers
• RT-1: Robotics Transformer for Real-World Control at Scale
• Towards Generalist Robots: A Promising Paradigm via Generative Simulation