MIND: Empowering Mental Health Clinicians with Multimodal Data Insights through a Narrative Dashboard
2026/01/21 by Ruishi Zou, Shiyu Xu, Margaret E Morris +11 · 1 voice
Computer Science · #cs.HC
paper · pdf · doi:10.1145/3772318.3790529
Abstract
Advances in data collection enable the capture of rich patient-generated data: from passive sensing (e.g., wearables and smartphones) to active self-reports (e.g., cross-sectional surveys and ecological momentary assessments). Although prior research has demonstrated the utility of patient-generated data in mental healthcare, significant challenges remain in effectively presenting these data streams along with clinical data (e.g., clinical notes) for clinical decision-making. Through co-design sessions with five clinicians, we propose MIND, a large language model-powered dashboard designed to present clinically relevant multimodal data insights for mental healthcare. MIND presents multimodal insights through narrative text, complemented by charts communicating underlying data. Our user study (N=16) demonstrates that clinicians perceive MIND as a significant improvement over baseline methods, reporting improved performance to reveal hidden and clinically relevant data insights (p<.001) and support their decision-making (p=.004). Grounded in the study results, we discuss future research opportunities to integrate data narratives in broader clinical practices.
Citations
- Beyond the Clinic: A Large-Scale Evaluation of Augmenting EHR with Wearable Data for Diverse Health Prediction
- The Anatomy of a Personal Health Agent
- GLOSS: Group of LLMs for Open-Ended Sensemaking of Passive Sensing Data for Health and Wellbeing
- SensorLM: Learning the Language of Wearable Sensors
- Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers.
- AgentAda: Skill-Adaptive Data Analytics for Tailored Insight Discovery
- More Modality, More AI: Exploring Design Opportunities of AI-Based Multi-modal Remote Monitoring Technologies for Early Detection of Mental Health Sequelae in Youth Concussion Patients
- Designing Technologies for Value-based Mental Healthcare: Centering Clinicians' Perspectives on Outcomes Data Specification, Collection, and Use
- A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
- HealthPrism: A Visual Analytics System for Exploring Children's Physical and Mental Health Profiles with Multimodal Data
- Chart2Vec: A Universal Embedding of Context-Aware Visualizations
- Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Care
- GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalization
- Large Language Models are Zero-Shot Reasoners
- AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts
- Towards an Automated SOAP Note: Classifying Utterances from Medical Conversations
- Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
- Anxiety Disorders in Primary Care: Prevalence, Impairment, Comorbidity, and Detection
- A Brief Measure for Assessing Generalized Anxiety Disorder
- Summarization from medical documents: a survey
- The Patient Health Questionnaire-2
- The PHQ-9
- Development and validation of brief measures of positive and negative affect: The PANAS scales.
- A Global Measure of Perceived Stress
Discussions
Related