2022/10/10 by Luca Schmidtke, Benjamin Hou, Schmidtke, Luca +5 · 1 citation
Computer Science · #Advanced Vision and Imaging #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Graphics (cs.GR) #Human Pose and Action Recognition #Video Surveillance and Tracking Methods
paper · pdf · doi:10.48550/arxiv.2210.04514
openalex publication_date 2022/10/10 · openalex created_date 2022/10/12 · openalex updated_date 2026/07/28
Inferring 3D human pose from 2D images is a challenging and long-standing problem in the field of computer vision with many applications including motion capture, virtual reality, surveillance or gait analysis for sports and medicine. We present preliminary results for a method to estimate 3D pose from 2D video containing a single person and a static background without the need for any manual landmark annotations. We achieve this by formulating a simple yet effective self-supervision task: our model is required to reconstruct a random frame of a video given a frame from another timepoint and a rendered image of a transformed human shape template. Crucially for optimisation, our ray casting based rendering pipeline is fully differentiable, enabling end to end training solely based on the reconstruction task.