LoRA-Tuned Large Language Models for Dementia Detection via Multi-View Speech-Derived Features

2026/06/26 by Jonghyeon Park, Olivier Jiyoun Jung, Myungwoo Oh
#cs.SD #cs.AI #cs.CL #cs.LG

paper · pdf

Abstract

Early detection of dementia enables timely intervention, and reflecting cognitive impairment, spontaneous speech offers a non-invasive screening modality. Conventional approaches often focus on a single representational dimension -- such as acoustic descriptors, pause modeling, automatic speech recognition (ASR) transcripts, or multimodal fusion -- limiting integrative reasoning across heterogeneous cognitive symptoms. We propose a low-rank adaptation (LoRA)-tuned large language model (LLM) that performs structured multi-view reasoning over four complementary speech-derived signals: ASR transcripts with pause markers, discourse-level topic cues, temporal fluency statistics, and phonological sequences. These cues are encoded within a unified prompt, enabling a single LLM to learn a coherent decision function without modality-specific encoders or late-stage fusion. On ADReSSo, our best model achieves an F1-score of 90.14%, and ablation confirms the complementary contribution of each view.

Citations

• LoRA: Low-Rank Adaptation of Large Language Models
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• Speech-Based Cognitive Screening: A Systematic Evaluation of LLM Adaptation Strategies
• Alzheimer's Dementia Recognition Using Acoustic, Lexical, Disfluency and Speech Pause Features Robust to Noisy Inputs
• Leveraging Cascaded Binary Classification and Multimodal Fusion for Dementia Detection through Spontaneous Speech
• Alzheimer's Dementia Detection Using Perplexity from Paired Large Language Models
• Early Dementia Detection Using Multiple Spontaneous Speech Prompts: The PROCESS Challenge