2020/09/01 by Necati Cihan Camgöz, Oscar Koller, Camgoz, Necati Cihan +5 · 5 citations
Computer Science · Psychology · #Hand Gesture Recognition Systems #Hearing Impairment and Communication #Human Pose and Action Recognition
paper · pdf · doi:10.48550/arxiv.2009.00299
Sign languages use multiple asynchronous information channels (articulators),\nnot just the hands but also the face and body, which computational approaches\noften ignore. In this paper we tackle the multi-articulatory sign language\ntranslation task and propose a novel multi-channel transformer architecture.\nThe proposed architecture allows both the inter and intra contextual\nrelationships between different sign articulators to be modelled within the\ntransformer network itself, while also maintaining channel specific\ninformation. We evaluate our approach on the RWTH-PHOENIX-Weather-2014T dataset\nand report competitive translation performance. Importantly, we overcome the\nreliance on gloss annotations which underpin other state-of-the-art approaches,\nthereby removing future need for expensive curated datasets.\n