Synthesizing Training Images for Boosting Human 3D Pose Estimation

2016/04/10 by Wenzheng Chen, Huan Wang, Chen, Wenzheng +16 · 66 citations
Computer Science · #3D pose estimation #Advanced Vision and Imaging #Artificial intelligence #Boosting (machine learning) #Computer science #Computer vision #Convolutional neural network #Ground truth #Human Pose and Action Recognition #Machine learning #Pattern recognition (psychology) #Pose #Task (project management) #Training set #Video Surveillance and Tracking Methods #cs.CV

paper · pdf · doi:10.48550/arxiv.1604.02703

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2016/04/10 · arxiv created 2017/01/05 · arxiv updated 2017/01/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Human 3D pose estimation from a single image is a challenging task with numerous applications. Convolutional Neural Networks (CNNs) have recently achieved superior performance on the task of 2D pose estimation from a single image, by training on images with 2D annotations collected by crowd sourcing. This suggests that similar success could be achieved for direct estimation of 3D poses. However, 3D poses are much harder to annotate, and the lack of suitable annotated training images hinders attempts towards end-to-end solutions. To address this issue, we opt to automatically synthesize training images with ground truth pose annotations. Our work is a systematic study along this road. We find that pose space coverage and texture diversity are the key ingredients for the effectiveness of synthetic training data. We present a fully automatic, scalable approach that samples the human pose space for guiding the synthesis procedure and extracts clothing textures from real images. Furthermore, we explore domain adaptation for bridging the gap between our synthetic training images and real testing photos. We demonstrate that CNNs trained with our synthetic images out-perform those trained with real photos on 3D pose estimation tasks.

Cited by

• Automated X-ray Image Analysis for Cargo Security: Critical Review and\n Future Promise
• Self-supervised Learning of Motion Capture
• Ordinal Depth Supervision for 3D Human Pose Estimation
• Toward Accurate and Realistic Outfits Visualization with Attention to Details
• VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera
• FBI-Pose: Towards Bridging the Gap between 2D Images and 3D Human Poses using Forward-or-Backward Information
• Adversarial Appearance Learning in Augmented Cityscapes for Pedestrian Recognition in Autonomous Driving
• Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB
• Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• SampleAhead: Online Classifier-Sampler Communication for Learning from Synthesized Data
• Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose
• Towards Robust RGB-D Human Mesh Recovery
• Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach
• Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision
• Latent Space Representation for Shape Analysis and Learning
• Domain Adaptation from Synthesis to Reality in Single-model Detector for Video Smoke Detection
• Toward Characteristic-Preserving Image-based Virtual Try-On Network
• Distill Knowledge from NRSfM for Weakly Supervised 3D Pose Learning
• BodyNet: Volumetric Inference of 3D Human Body Shapes
• Learning from Synthetic Humans

Related

• Image-based Synthesis for Deep 3D Human Pose Estimation
• MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
• Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision
• Learning camera viewpoint using CNN to improve 3D body pose estimation
• A Dual-Source Approach for 3D Human Pose Estimation from a Single Image