2025/12/17 by Xu, Youmin, Guo, Mengxi, Zhao, Shijie +4
Computer Science · #Advanced Data Compression Techniques #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Multimedia (cs.MM) #Speech and Audio Processing #Video Coding and Compression Technologies #electronic engineering #information engineering
paper · doi:10.48550/arxiv.2512.15262
openalex publication_date 2025/12/17 · openalex created_date 2025/12/19 · openalex updated_date 2026/07/28
Generative face video coding (GFVC) is vital for modern applications like video conferencing, yet existing methods primarily focus on video motion while neglecting the significant bitrate contribution of audio. Despite the well-established correlation between audio and lip movements, this cross-modal coherence has not been systematically exploited for compression. To address this, we propose an Audio-Visual Cross-Modal Compression (AVCC) framework that jointly compresses audio and video streams. Our framework extracts motion information from video and tokenizes audio features, then aligns them through a unified audio-video diffusion process. This allows synchronized reconstruction of both modalities from a shared representation. In extremely low-rate scenarios, AVCC can even reconstruct one modality from the other. Experiments show that AVCC significantly outperforms the Versatile Video Coding (VVC) standard and state-of-the-art GFVC schemes in rate-distortion performance, paving the way for more efficient multimodal communication systems.