Detection of social signals for recognizing engagement in human-robot\n interaction

2017/09/29 by Divesh Lala, Lala, Divesh, Koji Inoue +5
Computer Science · Psychology · #FOS: Computer and information sciences #Gaze Tracking and Assistive Technology #Human-Computer Interaction (cs.HC) #Multimodal Machine Learning Applications #Robotics (cs.RO) #Social Robot Interaction and HRI #Speech and dialogue systems

paper · pdf · doi:10.48550/arxiv.1709.10257

openalex publication_date 2017/09/29 · openalex created_date 2022/10/02 · openalex updated_date 2026/07/28

Abstract

Detection of engagement during a conversation is an important function of\nhuman-robot interaction. The level of user engagement can influence the\ndialogue strategy of the robot. Our motivation in this work is to detect\nseveral behaviors which will be used as social signal inputs for a real-time\nengagement recognition model. These behaviors are nodding, laughter, verbal\nbackchannels and eye gaze. We describe models of these behaviors which have\nbeen learned from a large corpus of human-robot interactions with the android\nrobot ERICA. Input data to the models comes from a Kinect sensor and a\nmicrophone array. Using our engagement recognition model, we can achieve\nreasonable performance using the inputs from automatic social signal detection,\ncompared to using manual annotation as input.\n

Related

• Explorations in engagement for humans and robots
• Robust Understanding of Human-Robot Social Interactions through Multimodal Distillation
• Using Vision-Language Models as Proxies for Social Intelligence in Human-Robot Interaction
• Forecasting Nonverbal Social Signals during Dyadic Interactions with Generative Adversarial Neural Networks
• ERR@HRI 2.0 Challenge: Multimodal Detection of Errors and Failures in Human-Robot Conversations