Transformer-XL for Long Sequence Tasks in Robotic Learning from Demonstration

2024/05/24 by Gao Tianci, Tianci, Gao · 2 citations
Engineering · #FOS: Computer and information sciences #Robotics (cs.RO) #Robotics and Automated Systems #Robotics and Sensor-Based Localization

paper · pdf · doi:10.48550/arxiv.2405.15562

openalex publication_date 2024/05/24 · openalex created_date 2024/05/28 · openalex updated_date 2026/07/28

Abstract

This paper presents an innovative application of Transformer-XL for long sequence tasks in robotic learning from demonstrations (LfD). The proposed framework effectively integrates multi-modal sensor inputs, including RGB-D images, LiDAR, and tactile sensors, to construct a comprehensive feature vector. By leveraging the advanced capabilities of Transformer-XL, particularly its attention mechanism and position encoding, our approach can handle the inherent complexities and long-term dependencies of multi-modal sensory data. The results of an extensive empirical evaluation demonstrate significant improvements in task success rates, accuracy, and computational efficiency compared to conventional methods such as Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNNs). The findings indicate that the Transformer-XL-based framework not only enhances the robot's perception and decision-making abilities but also provides a robust foundation for future advancements in robotic learning from demonstrations.

Cited by

• SAGE:State-Aware Guided End-to-End Policy for Multi-Stage Sequential Tasks via Hidden Markov Decision Process
• MTIL: Encoding Full History with Mamba for Temporal Imitation Learning

Related

• Robot Learning with Sensorimotor Pre-training
• PACT: Perception-Action Causal Transformer for Autoregressive Robotics Pre-Training
• RoboMorph: In-Context Meta-Learning for Robot Dynamics Modeling
• PLEX: Making the Most of the Available Data for Robotic Manipulation Pretraining
• Advances in Transformers for Robotic Applications: A Review