Autoregressive Sign Language Production: A Gloss-Free Approach with Discrete Representations

2023/09/21 by Hwang, Eui Jun, Lee, Huije, Park, Jong C. · 2 citations
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2309.12179

Abstract

Gloss-free Sign Language Production (SLP) offers a direct translation of spoken language sentences into sign language, bypassing the need for gloss intermediaries. This paper presents the Sign language Vector Quantization Network, a novel approach to SLP that leverages Vector Quantization to derive discrete representations from sign pose sequences. Our method, rooted in both manual and non-manual elements of signing, supports advanced decoding methods and integrates latent-level alignment for enhanced linguistic coherence. Through comprehensive evaluations, we demonstrate superior performance of our method over prior SLP methods and highlight the reliability of Back-Translation and Fréchet Gesture Distance as evaluation metrics.

Cited by

• Meaningful Pose-Based Sign Language Evaluation
• SLRTP2025 Sign Language Production Challenge: Methodology, Results, and Future Work

Related

• A Data-Driven Representation for Sign Language Production
• Non-Autoregressive Sign Language Production via Knowledge Distillation
• Disentangle and Regularize: Sign Language Production with Articulator-Based Disentanglement and Channel-Aware Regularization
• T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Text
• Text2Sign Diffusion: A Generative Approach for Gloss-Free Sign Language Production