Predicting Depression Severity by Multi-Modal Feature Engineering and Fusion

2017/11/29 by Samareh, Aven, Jin, Yan, Wang, Zhangyang +2
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.1711.11155

Abstract

We present our preliminary work to determine if patient's vocal acoustic, linguistic, and facial patterns could predict clinical ratings of depression severity, namely Patient Health Questionnaire depression scale (PHQ-8). We proposed a multi modal fusion model that combines three different modalities: audio, video , and text features. By training over AVEC 2017 data set, our proposed model outperforms each single modality prediction model, and surpasses the data set baseline with ice margin.

Related

• Depression Severity Estimation from Multiple Modalities
• Speech as a Multimodal Digital Phenotype for Multi-Task LLM-based Mental Health Prediction
• Cost-effective Models for Detecting Depression from Speech
• It Hears, It Sees too: Multi-Modal LLM for Depression Detection By Integrating Visual Understanding into Audio Language Models
• Depression Detection and Analysis using Large Language Models on Textual and Audio-Visual Modalities