2021/09/01 by Andreas Voskou, Konstantinos P. Panousis, Voskou, Andreas +8 · 3 citations
Computer Science · Engineering · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hand Gesture Recognition Systems #Human Pose and Action Recognition #Machine Learning (cs.LG) #Robot Manipulation and Learning #cs.CL #cs.LG
paper · pdf · doi:10.48550/arxiv.2109.13318
In Proceedings of ICCV 2021
openalex publication_date 2021/09/01 · arxiv created 2021/10/01 · arxiv updated 2021/10/04 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Automating sign language translation (SLT) is a challenging real world application. Despite its societal importance, though, research progress in the field remains rather poor. Crucially, existing methods that yield viable performance necessitate the availability of laborious to obtain gloss sequence groundtruth. In this paper, we attenuate this need, by introducing an end-to-end SLT model that does not entail explicit use of glosses; the model only needs text groundtruth. This is in stark contrast to existing end-to-end models that use gloss sequence groundtruth, either in the form of a modality that is recognized at an intermediate model stage, or in the form of a parallel output process, jointly trained with the SLT model. Our approach constitutes a Transformer network with a novel type of layers that combines: (i) local winner-takes-all (LWTA) layers with stochastic winner sampling, instead of conventional ReLU layers, (ii) stochastic weights with posterior distributions estimated via variational inference, and (iii) a weight compression technique at inference time that exploits estimated posterior variance to perform massive, almost lossless compression. We demonstrate that our approach can reach the currently best reported BLEU-4 score on the PHOENIX 2014T benchmark, but without making use of glosses for model training, and with a memory footprint reduced by more than 70%.