Transformer-based Action recognition in hand-object interacting scenarios

2022/10/20 by Cho, Hoseong, Baek, Seungryul
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2210.11387

Abstract

This report describes the 2nd place solution to the ECCV 2022 Human Body, Hands, and Activities (HBHA) from Egocentric and Multi-view Cameras Challenge: Action Recognition. This challenge aims to recognize hand-object interaction in an egocentric view. We propose a framework that estimates keypoints of two hands and an object with a Transformer-based keypoint estimator and recognizes actions based on the estimated keypoints. We achieved a top-1 accuracy of 87.19% on the testset.

Related

• Human Action Recognition in Egocentric Perspective Using 2D Object and Hands Pose
• Interaction Region Visual Transformer for Egocentric Action Anticipation
• Egocentric Hand Track and Object-based Human Action Recognition
• Egocentric Hand-object Interaction Detection
• EgoPCA: A New Framework for Egocentric Hand-Object Interaction Understanding