Low-Rank Head Avatar Personalization with Registers

2025/06/02 by Sai Tanmay Reddy Chakkera, Chakkera, Sai Tanmay Reddy, Aggelina Chatziagapi +9
Computer Science · Engineering · #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #Distributed and Parallel Computing Systems #FOS: Computer and information sciences #Human Motion and Animation

paper · pdf · doi:10.48550/arxiv.2506.01935

openalex publication_date 2025/06/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce a novel method for low-rank personalization of a generic model for head avatar generation. Prior work proposes generic models that achieve high-quality face animation by leveraging large-scale datasets of multiple identities. However, such generic models usually fail to synthesize unique identity-specific details, since they learn a general domain prior. To adapt to specific subjects, we find that it is still challenging to capture high-frequency facial details via popular solutions like low-rank adaptation (LoRA). This motivates us to propose a specific architecture, a Register Module, that enhances the performance of LoRA, while requiring only a small number of parameters to adapt to an unseen identity. Our module is applied to intermediate features of a pre-trained model, storing and re-purposing information in a learnable 3D feature space. To demonstrate the efficacy of our personalization method, we collect a dataset of talking videos of individuals with distinctive facial details, such as wrinkles and tattoos. Our approach faithfully captures unseen faces, outperforming existing methods quantitatively and qualitatively. We will release the code, models, and dataset to the public.

Citations

• PGC: Physics-Based Gaussian Cloth from a Single Pose
• InsTaG: Learning Personalized 3D Talking Head from Few-Second Video
• LAM: Large Avatar Model for One-shot Animatable Gaussian Head
• Generalizable and Animatable Gaussian Head Avatar
• Learning Frame-Wise Emotion Intensity for Audio-Driven Talking-Head Generation
• JEAN: Joint Expression and Audio-guided NeRF-based Talking Face Generation
• TalkLoRA: Low-Rank Adaptation for Speech-Driven Animation
• GaussianTalker: Real-Time High-Fidelity Talking Head Synthesis with Audio-Driven 3D Gaussian Splatting
• VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time
• MI-NeRF: Learning a Single Face NeRF from Multiple Identities
• Portrait4D-v2: Pseudo Multi-View Data Creates Better 4D Head Synthesizer
• Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Image
• SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting
• One2Avatar: Generative Implicit Head Avatar For Few-shot User Adaptation
• DoRA: Weight-Decomposed Low-Rank Adaptation
• GPAvatar: Generalizable and Precise Head Avatar from Image(s)
• MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatar
• HeadGaS: Real-Time Animatable Head Avatars via 3D Gaussian Splatting
• GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivation
• GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians
• FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embedding
• DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars
• Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Data
• Detecting Deepfakes Without Seeing Any
• Vision Transformers Need Registers
• 3D Gaussian Splatting for Real-Time Radiance Field Rendering
• 3D Gaussian Splatting for Real-Time Radiance Field Rendering
• My3DGen: A Scalable Personalized 3D Generative Model
• DINOv2: Learning Robust Visual Features without Supervision
• Adding Conditional Control to Text-to-Image Diffusion Models
• Adding Conditional Control to Text-to-Image Diffusion Models
• MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptation
• Instant Volumetric Head Avatars
• MARLIN: Masked Autoencoder for facial video Representation LearnINg
• VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution
• Efficient Geometry-aware 3D Generative Adversarial Networks
• LoRA: Low-Rank Adaptation of Large Language Models
• BEiT: BERT Pre-Training of Image Transformers
• Pivotal Tuning for Latent-based Editing of Real Images
• Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation
• WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition
• Portrait Neural Radiance Fields from a Single Image
• Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction
• Learned Initializations for Optimizing Coordinate-Based Neural Representations
• D-NeRF: Neural Radiance Fields for Dynamic Scenes
• Nerfies: Deformable Neural Radiance Fields
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• First Order Motion Model for Image Animation
• Realistic Speech-Driven Facial Animation with GANs
• Parameter-Efficient Transfer Learning for NLP
• Deep Video Portraits
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• Attention Is All You Need
• Memory Networks
• Long Short-Term Memory
• On the shape of a set of points in the plane
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priors
• PEGASUS: Personalized Generative 3D Avatars with Composable Attributes
• Audio-Driven Universal Gaussian Head Avatars
• PAV: Personalized Head Avatar from Unstructured Video Collection
• One2Avatar: Generative Implicit Head Avatar For Few-shot User Adaptation