2023/05/04 by Zhou'an Zhu, Xin Li, Zhou'an_Zhu +11 · 4 citations
Arts and Humanities · Health Professions · Medicine · #Computer Vision and Pattern Recognition (cs.CV) #Computers and Society (cs.CY) #Empathy and Medical Education #FOS: Computer and information sciences #Film in Education and Therapy #Media Influence and Health
paper · pdf · doi:10.48550/arxiv.2305.02842
openalex publication_date 2023/05/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Although empathic interaction between counselor and client is fundamental to success in the psychotherapeutic process, there are currently few datasets to aid a computational approach to empathy understanding. In this paper, we construct a multimodal empathy dataset collected from face-to-face psychological counseling sessions. The dataset consists of 771 video clips. We also propose three labels (i.e., expression of experience, emotional reaction, and cognitive reaction) to describe the degree of empathy between counselors and their clients. Expression of experience describes whether the client has expressed experiences that can trigger empathy, and emotional and cognitive reactions indicate the counselor's empathic reactions. As an elementary assessment of the usability of the constructed multimodal empathy dataset, an interrater reliability analysis of annotators' subjective evaluations for video clips is conducted using the intraclass correlation coefficient and Fleiss' Kappa. Results prove that our data annotation is reliable. Furthermore, we conduct empathy prediction using three typical methods, including the tensor fusion network, the sentimental words aware fusion network, and a simple concatenation model. The experimental results show that empathy can be well predicted on our dataset. Our dataset is available for research purposes.