3D Human Pose Estimation with Spatial and Temporal Transformers

2021/03/18 by Ce Zheng, Sijie Zhu, Zheng, Ce +10 · 40 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Hand Gesture Recognition Systems #Human Pose and Action Recognition #Human-Computer Interaction (cs.HC) #Video Surveillance and Tracking Methods #cs.AI #cs.CV #cs.HC

paper · pdf · doi:10.48550/arxiv.2103.10455

ICCV 2021

openalex publication_date 2021/03/18 · arxiv created 2021/08/22 · arxiv updated 2021/08/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Transformer architectures have become the model of choice in natural language processing and are now being introduced into computer vision tasks such as image classification, object detection, and semantic segmentation. However, in the field of human pose estimation, convolutional architectures still remain dominant. In this work, we present PoseFormer, a purely transformer-based approach for 3D human pose estimation in videos without convolutional architectures involved. Inspired by recent developments in vision transformers, we design a spatial-temporal transformer structure to comprehensively model the human joint relations within each frame as well as the temporal correlations across frames, then output an accurate 3D human pose of the center frame. We quantitatively and qualitatively evaluate our method on two popular and standard benchmark datasets: Human3.6M and MPI-INF-3DHP. Extensive experiments show that PoseFormer achieves state-of-the-art performance on both datasets. Code is available at \urlhttps://github.com/zczcwh/PoseFormer

Cited by

• PoseMoE: Mixture-of-Experts Network for Monocular 3D Human Pose Estimation
• RUMPL: Ray-Based Transformers for Universal Multi-View 2D to 3D Human Pose Lifting
• FastDDHPose: Towards Unified, Efficient, and Disentangled 3D Human Pose Estimation
• QuaMo: Quaternion Motions for Vision-based 3D Human Kinematics Capture
• HGFreNet: Hop-hybrid GraphFomer for 3D Human Pose Estimation with Trajectory Consistency in Frequency Domain
• PRGCN: A Graph Memory Network for Cross-Sequence Pattern Reuse in 3D Human Pose Estimation
• Robotic Classification of Divers' Swimming States using Visual Pose Keypoints as IMUs
• Paving the Way Towards Kinematic Assessment Using Monocular Video: A Preclinical Benchmark of State-of-the-Art Deep-Learning-Based 3D Human Pose Estimators Against Inertial Sensors in Daily Living Activities
• Bimanual 3D Hand Motion and Articulation Forecasting in Everyday Images
• H₂OT: Hierarchical Hourglass Tokenizer for Efficient Video Pose Transformers
• Efficient Diffusion-Based 3D Human Pose Estimation with Hierarchical Temporal Pruning
• HyperDiff: Hypergraph Guided Diffusion Model for 3D Human Pose Estimation
• Human-in-Context: Unified Cross-Domain 3D Human Motion Modeling via In-Context Learning
• AugLift: Depth-Aware Input Reparameterization Improves Domain Generalization in 2D-to-3D Pose Lifting
• VesselRW: Weakly Supervised Subcutaneous Vessel Segmentation via Learned Random Walk Propagation
• DualResolution Residual Architecture with Artifact Suppression for Melanocytic Lesion Segmentation
• PyCAT4: A Hierarchical Vision Transformer-based Framework for 3D Human Pose Estimation
• Deeply Dual Supervised learning for melanoma recognition
• StarPose: 3D Human Pose Estimation via Spatial-Temporal Autoregressive Diffusion
• KASportsFormer: Kinematic Anatomy Enhanced Transformer for 3D Human Pose Estimation on Short Sports Scene Video
• PUMPS: Skeleton-Agnostic Point-based Universal Motion Pre-Training for Synthesis in Human Motion Tasks
• A Structure-aware and Motion-adaptive Framework for 3D Human Pose Estimation with Mamba
• A Unified Transformer-Based Framework with Pretraining For Whole Body Grasping Motion Generation
• Frequency-Semantic Enhanced Variational Autoencoder for Zero-Shot Skeleton-based Action Recognition
• PoseGRAF: Geometric-Reinforced Adaptive Fusion for Monocular 3D Human Pose Estimation
• HDiffTG: A Lightweight Hybrid Diffusion-Transformer-GCN Architecture for 3D Human Pose Estimation
• Learning Pyramid-structured Long-range Dependencies for 3D Human Pose Estimation
• Spectral Compression Transformer with Line Pose Graph for Monocular 3D Human Pose Estimation
• Mamba-Driven Topology Fusion for Monocular 3D Human Pose Estimation
• PoseBench3D: A Cross-Dataset Analysis Framework for 3D Human Pose Estimation via Pose Lifting Networks
• Decoding Children's Gait Behavior
• 3D Human Pose Estimation via Spatial Graph Order Attention and Temporal Body Aware Transformer
• Dual-stream spatio-temporal GCN-transformer network for 3D human pose estimation
• Towards Ball Spin and Trajectory Analysis in Table Tennis Broadcast Videos via Physically Grounded Synthetic-to-Real Transfer
• Unsupervised Cross-Domain 3D Human Pose Estimation via Pseudo-Label-Guided Global Transforms
• CoMotion: Concurrent Multi-person 3D Motion
• H-MoRe: Learning Human-centric Motion Representation for Action Analysis
• Efficient 2D to Full 3D Human Pose Uplifting including Joint Rotations
• Benchmarking 3D Human Pose Estimation Models under Occlusions
• Multi-person Physics-based Pose Estimation for Combat Sports

Related

• CrossFormer: Cross Spatio-Temporal Transformer for 3D Human Pose Estimation
• ConvFormer: Parameter Reduction in Transformer Models for 3D Human Pose Estimation by Leveraging Dynamic Multi-Headed Convolutional Attention
• Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation
• PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation
• Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers