DR.EHR: Dense Retrieval for Electronic Health Record with Knowledge Injection and Synthetic Data

2025/07/24 by Zhengyun Zhao, Huaiyuan Ying, Zhao, Zhengyun +5
Biochemistry, Genetics and Molecular Biology · Computer Science · #Artificial Intelligence (cs.AI) #Biomedical Text Mining and Ontologies #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning in Healthcare

paper · pdf · doi:10.48550/arxiv.2507.18583

openalex publication_date 2025/07/24 · openalex created_date 2025/10/16 · openalex updated_date 2026/07/28

Abstract

Electronic Health Records (EHRs) are pivotal in clinical practices, yet their retrieval remains a challenge mainly due to semantic gap issues. Recent advancements in dense retrieval offer promising solutions but existing models, both general-domain and biomedical-domain, fall short due to insufficient medical knowledge or mismatched training corpora. This paper introduces DR.EHR, a series of dense retrieval models specifically tailored for EHR retrieval. We propose a two-stage training pipeline utilizing MIMIC-IV discharge summaries to address the need for extensive medical knowledge and large-scale training data. The first stage involves medical entity extraction and knowledge injection from a biomedical knowledge graph, while the second stage employs large language models to generate diverse training data. We train two variants of DR.EHR, with 110M and 7B parameters, respectively. Evaluated on the CliniQ benchmark, our models significantly outperforms all existing dense retrievers, achieving state-of-the-art results. Detailed analyses confirm our models' superiority across various match and query types, particularly in challenging semantic matches like implication and abbreviation. Ablation studies validate the effectiveness of each pipeline component, and supplementary experiments on EHR QA datasets demonstrate the models' generalizability on natural language questions, including complex ones with multiple entities. This work significantly advances EHR retrieval, offering a robust solution for clinical applications.

Citations

• GENIE: Generative Note Information Extraction model for structuring EHR data
• Lessons Learned on Information Retrieval in Electronic Health Records: A Comparison of Embedding Models and Pooling Strategies
• Lessons learned on information retrieval in electronic health records: a comparison of embedding models and pooling strategies
• NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models
• Onco-Retriever: Generative Classifier for Retrieval of EHR Records in Oncology
• CoRTEx: Contrastive Learning for Representing Terms via Explanations with Applications on Constructing Biomedical Knowledge Graphs
• Towards General Text Embeddings with Multi-stage Contrastive Learning
• Typos-aware Bottlenecked Pre-Training for Robust Dense Retrieval
• Promptagator: Few-shot Dense Retrieval From 8 Examples
• Review of Natural Language Processing in Pharmacology
• Automatic Biomedical Term Clustering by Learning Fine-grained Term Representations
• BIOS: An Algorithmically Generated Biomedical Knowledge Graph
• Text and Code Embeddings by Contrastive Pre-Training
• Finetuned Language Models Are Zero-Shot Learners
• LoRA: Low-Rank Adaptation of Large Language Models
• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
• CODER: Knowledge infused cross-lingual medical term embedding for term normalization
• Self-Alignment Pretraining for Biomedical Entity Representations
• UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual\n Embeddings Using the Unified Medical Language System Metathesaurus
• Patient Cohort Retrieval using Transformer Language Models
• Dense Passage Retrieval for Open-Domain Question Answering
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• Document Expansion by Query Prediction
• Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning
• emrQA: A Large Corpus for Question Answering on Electronic Medical Records

Related

• Lessons Learned on Information Retrieval in Electronic Health Records: A Comparison of Embedding Models and Pooling Strategies
• CliniQ: A Multi-faceted Benchmark for Electronic Health Record Retrieval with Semantic Match Assessment
• Lessons learned on information retrieval in electronic health records: a comparison of embedding models and pooling strategies
• Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs
• EHR-R1: A Reasoning-Enhanced Foundational Language Model for Electronic Health Record Analysis