MOTOR: A Time-To-Event Foundation Model For Structured Medical Records

2023/01/09 by Ethan Steinberg, Steinberg, Ethan, Jason Fries +5 · 24 citations
Computer Science · Health Professions · #Electronic Health Records Systems #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare #Medical Coding and Health Information

paper · pdf · doi:10.48550/arxiv.2301.03150

openalex publication_date 2023/01/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present a self-supervised, time-to-event (TTE) foundation model called MOTOR (Many Outcome Time Oriented Representations) which is pretrained on timestamped sequences of events in electronic health records (EHR) and health insurance claims. TTE models are used for estimating the probability distribution of the time until a specific event occurs, which is an important task in medical settings. TTE models provide many advantages over classification using fixed time horizons, including naturally handling censored observations, but are challenging to train with limited labeled data. MOTOR addresses this challenge by pretraining on up to 55M patient records (9B clinical events). We evaluate MOTOR's transfer learning performance on 19 tasks, across 3 patient databases (a private EHR system, MIMIC-IV, and Merative claims data). Task-specific models adapted from MOTOR improve time-dependent C statistics by 4.6% over state-of-the-art, improve label efficiency by up to 95% ,and are more robust to temporal distributional shifts. We further evaluate cross-site portability by adapting our MOTOR foundation model for six prediction tasks on the MIMIC-IV dataset, where it outperforms all baselines. MOTOR is the first foundation model for medical TTE predictions and we release a 143M parameter pretrained model for research use at [redacted URL].

Cited by

• Fusion or Confusion? Multimodal Complexity Is Not All You Need
• Medical world models in healthcare: foundations, applications, and challenges for trustworthy clinical translation
• Toward Scalable Early Cancer Detection: Evaluating EHR-Based Predictive Models Against Traditional Screening Criteria
• Count-Based Approaches Remain Strong: A Benchmark Against Transformer and LLM Pipelines on Structured EHR
• Serialized EHR make for good text representations
• Learning Survival Models with Right-Censored Reporting Delays
• Building the EHR Foundation Model via Next Event Prediction
• Beyond the Clinic: A Large-Scale Evaluation of Augmenting EHR with Wearable Data for Diverse Health Prediction
• Learning the natural history of human disease with generative transformers
• CEHR-XGPT: A Scalable Multi-Task Foundation Model for Electronic Health Records
• Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation models
• Time-Aware Attention for Enhanced Electronic Health Records Modeling
• Fusion is not one-size-fits-all: Cross-Modal Representation Alignment for Time-to-Event Modeling
• Federated Timeline Synthesis: Scalable and Private Methodology For Model Training and Deployment
• EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Records
• Position: Stop Chasing the C-index when Evaluating Survival Analysis Models
• Intercept Cancer: Cancer Pre-Screening with Large Scale Healthcare Foundation Models
• Will Large Language Models Transform Clinical Prediction?
• FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records
• SurvUnc: A Meta-Model Based Uncertainty Quantification Framework for Survival Analysis
• Federated generative event models for tokenized electronic health records
• LLMs can construct powerful representations and streamline sample-efficient supervised learning
• Temporal Entailment Pretraining for Clinical Language Models over EHR Data
• A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMs
• Foundation models for electronic health records: representation dynamics and transferability

Related

• LLM4EHR: Aligning Clinical Time Series with Medical Event Sequences via Large Language Models
• Foundation Models for Clinical Records at Health System Scale
• Learning Hierarchical Representations of Electronic Health Records for Clinical Outcome Prediction
• Building the EHR Foundation Model via Next Event Prediction
• Time-Aware Attention for Enhanced Electronic Health Records Modeling