2022/12/18 by Tiantian Feng, Shrikanth Narayanan, Feng, Tiantian +1
Psychology · #Audio and Speech Processing (eess.AS) #Communication in Education and Healthcare #Emotion and Mood Recognition #FOS: Computer and information sciences #FOS: Electrical engineering #Multimedia (cs.MM) #Sound (cs.SD) #Team Dynamics and Performance #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2212.09090
openalex publication_date 2022/12/18 · openalex created_date 2023/01/04 · openalex updated_date 2026/07/28
Interpersonal spoken communication is central to human interaction and the exchange of information. Such interactive processes involve not only speech and spoken language but also non-verbal cues such as hand gestures, facial expressions, and nonverbal vocalization, that are used to express feelings and provide feedback. These multimodal communication signals carry a variety of information about the people: traits like gender and age as well as about physical and psychological states and behavior. This work uses wearable multimodal sensors to investigate interpersonal communication behaviors focusing on speaking patterns among healthcare providers with a focus on nurses. We analyze longitudinal data collected from 99 nurses in a large hospital setting over ten weeks. The results indicate that speaking pattern differences across shift schedules and working units. Moreover, results show that speaking patterns combined with physiological measures can be used to predict affect measures and life satisfaction scores. The implementation of this work can be accessed at https://github.com/usc-sail/tiles-audio-arousal.