Integrating Text and Time-Series into (Large) Language Models to Predict Medical Outcomes

2025/09/17 by Iyadh Ben Cheikh Larbi, Ajay Madhavan Ravichandran, Larbi, Iyadh Ben Cheikh +5
Computer Science · Medicine · #Machine Learning in Healthcare #Topic Modeling #Artificial Intelligence in Healthcare and Education

paper · pdf · doi:10.48550/arxiv.2509.13696

Abstract

Large language models (LLMs) excel at text generation, but their ability to handle clinical classification tasks involving structured data, such as time series, remains underexplored. In this work, we adapt instruction-tuned LLMs using DSPy-based prompt optimization to process clinical notes and structured EHR inputs jointly. Our results show that this approach achieves performance on par with specialized multimodal systems while requiring less complexity and offering greater adaptability across tasks.

Citations

• DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
• Large Language Models are Few-Shot Health Learners
• Frozen Language Model Helps ECG Zero-Shot Learning
• Literature-Augmented Clinical Outcome Prediction
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration
• Using Clinical Notes with Time Series Data for ICU Management
• Publicly Available Clinical BERT Embeddings
• Practical Text Classification With Large Pre-Trained Language Models

Related

• SPeC: A Soft Prompt-Based Calibration on Performance Variability of Large Language Model in Clinical Notes Summarization
• Will Large Language Models Transform Clinical Prediction?
• Prompting Large Language Models for Zero-Shot Clinical Prediction with Structured Longitudinal Electronic Health Record Data
• Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Prediction
• Fine-Tuning LLMs on Small Medical Datasets: Text Classification and Normalization Effectiveness on Cardiology reports and Discharge records