Monocular 3D Hand Pose Estimation with Implicit Camera Alignment

2025/06/10 by Christos Pantazopoulos, Pantazopoulos, Christos, Spyridon Thermos +3
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Graphics (cs.GR) #Hand Gesture Recognition Systems #Human Pose and Action Recognition #Image and Video Processing (eess.IV) #Image and Video Stabilization #Machine Learning (cs.LG) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2506.11133

openalex publication_date 2025/06/10 · openalex created_date 2025/10/11 · openalex updated_date 2026/07/28

Abstract

Estimating the 3D hand articulation from a single color image is an important problem with applications in Augmented Reality (AR), Virtual Reality (VR), Human-Computer Interaction (HCI), and robotics. Apart from the absence of depth information, occlusions, articulation complexity, and the need for camera parameters knowledge pose additional challenges. In this work, we propose an optimization pipeline for estimating the 3D hand articulation from 2D keypoint input, which includes a keypoint alignment step and a fingertip loss to overcome the need to know or estimate the camera parameters. We evaluate our approach on the EgoDexter and Dexter+Object benchmarks to showcase that it performs competitively with the state-of-the-art, while also demonstrating its robustness when processing "in-the-wild" images without any prior camera knowledge. Our quantitative analysis highlights the sensitivity of the 2D keypoint estimation accuracy, despite the use of hand priors. Code is available at the project page https://cpantazop.github.io/HandRepo/

Citations

• Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild
• Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data
• FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from\n Single RGB Images
• MediaPipe: A Framework for Building Perception Pipelines
• Expressive Body Capture: 3D Hands, Face, and Body from a Single Image
• Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via\n Neural Rendering
• 3D Hand Shape and Pose Estimation from a Single RGB Image
• End-to-end Hand Mesh Recovery from a Monocular RGB Image
• 3D Hand Shape and Pose from Images in the Wild
• Monocular Total Capture: Posing Face, Body, and Hands in the Wild
• Hand Pose Estimation via Latent 2.5D Heatmap Regression
• Cross-modal Deep Variational Hand Pose Estimation
• Using a single RGB frame for real time 3D hand pose estimation in the\n wild
• GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB
• Learning to Estimate 3D Hand Pose from Single RGB Images
• Hand Keypoint Detection in Single Images using Multiview Bootstrapping
• Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input

Related

• An End-to-end Framework for Unconstrained Monocular 3D Hand Pose Estimation
• Efficient Annotation and Learning for 3D Hand Pose Estimation: A Survey
• Articulated Hand Pose Estimation Review
• Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interaction
• Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data