Sensing, Social, and Motion Intelligence in Embodied Navigation: A Comprehensive Survey

2025/08/21 by Chaoran Xiong, Yulong Huang, Xiong, Chaoran +11 · 1 citation
Psychology · Computer Science · #Social Robot Interaction and HRI #Action Observation and Synchronization #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2508.15354

Abstract

Embodied navigation (EN) advances traditional navigation by enabling robots to perform complex egocentric tasks through sensing, social, and motion intelligence. In contrast to classic methodologies that rely on explicit localization and pre-defined maps, EN leverages egocentric perception and human-like interaction strategies. This survey introduces a comprehensive EN formulation structured into five stages: Transition, Observation, Fusion, Reward-policy construction, and Action (TOFRA). The TOFRA framework serves to synthesize the current state of the art, provide a critical review of relevant platforms and evaluation metrics, and identify critical open research challenges. A list of studies is available at https://github.com/Franky-X/Awesome-Embodied-Navigation.

Citations

• A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI
• THE-SEAN: A Heart Rate Variation-Inspired Temporally High-Order Event-Based Visual Odometry with Self-Supervised Spiking Event Accumulation Networks
• Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks
• NavRAG: Generating User Demand Instructions for Embodied Navigation through Retrieval-Augmented LLM
• EmbodiedRAG: Dynamic 3D Scene Graph Retrieval for Efficient and Scalable Robot Task Planning
• X-MOBILITY: End-To-End Generalizable Navigation via World Modeling
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents
• GRUtopia: Dream General Robots in a City at Scale
• UMI on Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllers
• HiLMa-Res: A General Hierarchical Framework via Residual RL for Combining Quadrupedal Locomotion and Manipulation
• Distributed multi-robot potential-field-based exploration with submap-based mapping and noise-augmented strategy
• WoCoCo: Learning Whole-Body Humanoid Control with Sequential Contacts
• UCDNet: Multi-UAV Collaborative 3D Object Detection Network by Reliable Feature Mapping
• DrEureka: Language Model Guided Sim-To-Real Transfer
• A Survey on Vision-Language-Action Models for Embodied AI
• Scenarios for the Transition to AGI
• NeuPAN: Direct Point Robot Navigation with End-to-End Model-based Learning
• Learning to walk in confined spaces using 3D representation
• Whole-body Humanoid Robot Locomotion with Human Reference
• Explicit Interaction for Fusion-Based Place Recognition
• Pedipulate: Enabling Manipulation Skills using a Quadruped Robot's Leg
• Tight Fusion of Events and Inertial Measurements for Direct Velocity Estimation
• SNI-SLAM: Semantic Neural Implicit SLAM
• Exploitation-Guided Exploration for Semantic Embodied Navigation
• Advances in Embodied Navigation Using Large Language Models: A Survey
• Learning Agility and Adaptive Legged Locomotion via Curricular Hindsight Reinforcement Learning
• Resilient Legged Local Navigation: Learning to Traverse with Compromised Perception End-to-End
• End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenon
• Object Goal Navigation with Recursive Implicit Maps
• Ground-VIO: Monocular Visual-Inertial Odometry with Online Calibration of Camera-Ground Geometric Parameters
• EgoHumans: An Egocentric 3D Multi-Human Benchmark
• Modality-invariant Visual Odometry for Embodied Vision
• R²Former: Unified Retrieval and Reranking Transformer for Place Recognition
• GlueStick: Robust Image Matching by Sticking Points and Lines Together
• NeRF-LOAM: Neural Implicit Representation for Large-Scale Incremental LiDAR Odometry and Mapping
• Real-World Humanoid Locomotion with Reinforcement Learning
• BEVPlace: Learning LiDAR-based Place Recognition using Bird's Eye View Images
• A Comprehensive Review on Autonomous Navigation
• BEVBert: Multimodal Map Pre-training for Language-guided Navigation
• Exploiting Proximity-Aware Tasks for Embodied Social Navigation
• 3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification
• Visual Language Maps for Robot Navigation
• Advanced Skills by Learning Locomotion and Local Navigation End-to-End
• Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps
• PL-EVIO: Robust Monocular Event-based Visual Inertial Odometry with Point and Line Features
• RACER: Rapid Collaborative Exploration with a Decentralized Multi-UAV System
• SeqOT: A Spatial-Temporal Transformer Network for Place Recognition Using Sequential LiDAR Data
• Stronger Together: Air-Ground Robotic Collaboration Using Semantics
• ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings
• Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selection
• Online Learning of Reusable Abstract Models for Object Goal Navigation
• Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation
• E²(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition
• Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling
• AdaFusion: Visual-LiDAR Fusion with Adaptive Weights for Place Recognition
• SGoLAM: Simultaneous Goal Localization and Mapping for Multi-Object Goal Navigation
• Autonomous and Adaptive Navigation for Terrestrial-Aerial Bimodal Vehicles
• R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package
• The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation
• Image-Goal Navigation in Complex Environments via Modular Learning
• KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics
• Teaching Agents how to Map: Spatial Reasoning for Multi-Object Navigation
• Habitat 2.0: Training Home Assistants to Rearrange their Habitat
• SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
• Efficient and Robust LiDAR-Based End-to-End Navigation
• Collaborative Mapping of Archaeological Sites using multiple UAVs
• VTNet: Visual Transformer Network for Object Goal Navigation
• LVI-SAM: Tightly-coupled Lidar-Visual-Inertial Odometry via Smoothing and Mapping
• LoFTR: Detector-Free Local Feature Matching with Transformers
• A Survey of Embodied AI: From Simulators to Research Tasks
• A Trident Quaternion Framework for Inertial-based Navigation Part I: Rigid Motion Representation and Computation
• MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation
• Visual Navigation in Real-World Indoor Environments Using End-to-End Deep Reinforcement Learning
• FAST-LIO: A Fast, Robust LiDAR-inertial Odometry Package by Tightly-Coupled Iterated Kalman Filter
• Whole-Body MPC and Online Gait Sequence Generation for Wheeled-Legged Robots
• MARS: Mixed Virtual and Real Wearable Sensors for Human Activity Recognition with Multi-Domain Deep Learning Model
• Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agents
• Exploiting Scene-specific Features for Object Goal Navigation
• ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM
• EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection
• ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation
• Object Goal Navigation using Goal-Oriented Semantic Exploration
• LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping
• LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping
• Improving Vision-and-Language Navigation with Image-Text Pairs from the\n Web
• RoboTHOR: An Open Simulation-to-Real Embodied AI Platform
• SAPIEN: A SimulAted Part-based Interactive ENvironment
• Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training
• SuperGlue: Learning Feature Matching with Graph Neural Networks
• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
• SelfVIO: Self-Supervised Deep Monocular Visual-Inertial Odometry and Depth Estimation
• SelfVIO: Self-supervised deep monocular Visual–Inertial Odometry and depth estimation
• Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation
• DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames
• Estimating Lower Limb Kinematics using a Reduced Wearable Sensor Count
• LIC-Fusion: LiDAR-Inertial-Camera Odometry
• DeepVIO: Self-supervised Deep Learning of Monocular Visual Inertial Odometry using 3D Geometric Constraints
• iNavFIter: Next-Generation Inertial Navigation Computation Based on Functional Iteration
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Hierarchical Reinforcement Learning for Quadruped Locomotion
• KPConv: Flexible and Deformable Convolution for Point Clouds
• AI-IMU Dead-Reckoning
• Habitat: A Platform for Embodied AI Research
• Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
• Learning Exploration Policies for Navigation
• Selective Sensor Fusion for Neural Visual-Inertial Odometry
• Learning agile and dynamic motor skills for legged robots
• Occupancy Networks: Learning 3D Reconstruction in Function Space
• StructVIO : Visual-inertial Odometry with Structural Regularity of Man-made Environments
• Deep Continuous Fusion for Multi-Sensor 3D Object Detection
• Curiosity-driven Exploration for Mapless Navigation with Deep Reinforcement Learning
• End-to-End Learning of Driving Models with Surround-View Cameras and\n Route Planners
• SuperPoint: Self-Supervised Interest Point Detection and Description
• AI2-THOR: An Interactive 3D Environment for Visual AI
• Vision-and-Language Navigation: Interpreting visually-grounded\n navigation instructions in real environments
• CARLA: An Open Urban Driving Simulator
• Attention Is All You Need
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem
• NetVLAD: CNN architecture for weakly supervised place recognition
• Real-time Monocular Object SLAM
• Fully Convolutional Networks for Semantic Segmentation
• The invariant extended Kalman filter as a stable observer
• Distinctive Image Features from Scale-Invariant Keypoints
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• AUV Navigation and Localization: A Review

Cited by

• Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges

Related

• SocialNav: Training Human-Inspired Foundation Model for Socially-Aware Embodied Navigation
• Deep Learning for Embodied Vision Navigation: A Survey
• In Search of a Lost Metric: Human Empowerment as a Pillar of Socially Conscious Navigation
• A Survey on Human-aware Robot Navigation
• Advances in Embodied Navigation Using Large Language Models: A Survey