Zero shot health trajectory prediction using transformer

2024/09/19 by Paweł Renc, Yugang Jia, Anthony E. Samir +4 · 1 voice · 2 citations
Medicine · Computer Science · #COVID-19 diagnosis using AI #Artificial Intelligence in Healthcare and Education #Machine Learning in Healthcare

paper · pdf · doi:10.1038/s41746-024-01235-0

openalex publication_date 2024/09/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

Integrating modern machine learning and clinical decision-making has great promise for mitigating healthcare's increasing cost and complexity. We introduce the Enhanced Transformer for Health Outcome Simulation (ETHOS), a novel application of the transformer deep-learning architecture for analyzing high-dimensional, heterogeneous, and episodic health data. ETHOS is trained using Patient Health Timelines (PHTs)-detailed, tokenized records of health events-to predict future health trajectories, leveraging a zero-shot learning approach. ETHOS represents a significant advancement in foundation model development for healthcare analytics, eliminating the need for labeled data and model fine-tuning. Its ability to simulate various treatment pathways and consider patient-specific factors positions ETHOS as a tool for care optimization and addressing biases in healthcare delivery. Future developments will expand ETHOS' capabilities to incorporate a wider range of data types and data sources. Our work demonstrates a pathway toward accelerated AI development and deployment in healthcare.

Citations

• Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Events
• Large language models in medicine
• Exploiting Generative AI to Scale up Intelligent Tutoring Systems
• TRAM: Benchmarking Temporal Reasoning for Large Language Models
• Using sequences of life-events to predict human lives

Cited by

• Federated generative event models for tokenized electronic health records
• LLMs can construct powerful representations and streamline sample-efficient supervised learning

Discussions

• Fascinating idea: Tokenizing patient health trajectories to create a foundation model for predictive healthcare analytics. doi.org/10.1038/s417... [bsky, 0 points, 0 comments]

Related

• Zero-shot Medical Event Prediction Using a Generative Pre-trained Transformer on Electronic Health Records
• Patient Trajectory Prediction: Integrating Clinical Notes with Transformers
• Towards modeling evolving longitudinal health trajectories with a transformer-based deep learning model
• Foresight -- Generative Pretrained Transformer (GPT) for Modelling of Patient Timelines using EHRs
• Foundation Model of Electronic Medical Records for Adaptive Risk Estimation