Contextual Phenotyping of Pediatric Sepsis Cohort Using Large Language Models

2025/05/14 by Aditya Nagori, Nagori, Aditya, Matthew O. Wiens +14
Computer Science · #Applications (stat.AP) #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Biological sciences #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare #Quantitative Methods (q-bio.QM) #Text Readability and Simplification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2505.09805

openalex publication_date 2025/05/14 · openalex created_date 2025/10/15 · openalex updated_date 2026/07/28

Abstract

Clustering patient subgroups is essential for personalized care and efficient resource use. Traditional clustering methods struggle with high-dimensional, heterogeneous healthcare data and lack contextual understanding. This study evaluates Large Language Model (LLM) based clustering against classical methods using a pediatric sepsis dataset from a low-income country (LIC), containing 2,686 records with 28 numerical and 119 categorical variables. Patient records were serialized into text with and without a clustering objective. Embeddings were generated using quantized LLAMA 3.1 8B, DeepSeek-R1-Distill-Llama-8B with low-rank adaptation(LoRA), and Stella-En-400M-V5 models. K-means clustering was applied to these embeddings. Classical comparisons included K-Medoids clustering on UMAP and FAMD-reduced mixed data. Silhouette scores and statistical tests evaluated cluster quality and distinctiveness. Stella-En-400M-V5 achieved the highest Silhouette Score (0.86). LLAMA 3.1 8B with the clustering objective performed better with higher number of clusters, identifying subgroups with distinct nutritional, clinical, and socioeconomic profiles. LLM-based methods outperformed classical techniques by capturing richer context and prioritizing key features. These results highlight potential of LLMs for contextual phenotyping and informed decision-making in resource-limited settings.

Citations

• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval
• A Comprehensive Overview of Large Language Models
• LoRA: Low-Rank Adaptation of Large Language Models
• A Transformer-based Framework for Multivariate Time Series Representation Learning
• Language Models are Few-Shot Learners
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• Attention Is All You Need
• Towards A Rigorous Science of Interpretable Machine Learning
• Efficient Estimation of Word Representations in Vector Space
• Use of Ranks in One-Criterion Variance Analysis
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• MapReduce

Related

• Bayesian Learning of Clinically Meaningful Sepsis Phenotypes in Northern Tanzania
• Identification of Pediatric Sepsis Subphenotypes for Enhanced Machine Learning Predictive Performance: A Latent Profile Analysis
• Combining Domain-Specific Models and LLMs for Automated Disease Phenotyping from Survey Data
• Contextual Evaluation of Large Language Models for Classifying Tropical and Infectious Diseases
• Extending Machine Learning-Based Early Sepsis Detection to Different Demographics