Blendshapes GHUM: Real-time Monocular Facial Blendshape Prediction

2023/09/11 by Grishchenko, Ivan, Yan, Geng, Bazavan, Eduard Gabriel +5 · 1 citation
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2309.05782

Abstract

We present Blendshapes GHUM, an on-device ML pipeline that predicts 52 facial blendshape coefficients at 30+ FPS on modern mobile phones, from a single monocular RGB image and enables facial motion capture applications like virtual avatars. Our main contributions are: i) an annotation-free offline method for obtaining blendshape coefficients from real-world human scans, ii) a lightweight real-time model that predicts blendshape coefficients based on facial landmarks.

Cited by

• If They Disagree, Will You Conform? Exploring the Role of Robots' Value Awareness in a Decision-Making Task

Related

• RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatars
• HRM²Avatar: High-Fidelity Real-Time Mobile Avatars from Monocular Phone Scans
• FitMe: Deep Photorealistic 3D Morphable Model Avatars
• High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies
• Neural Volumetric Blendshapes: Computationally Efficient Physics-Based Facial Blendshapes