BioHiCL: Hierarchical Multi-Label Contrastive Learning for Biomedical Retrieval with MeSH Labels

2026/04/30 by Mengfei Lan, Lecheng Zheng, Halil Kilicoglu
Computer Science · #cs.AI #cs.IR

paper · pdf

Accepted by ACL 2026 (Oral)

arxiv created 2026/07/28 · arxiv updated 2026/07/30

Abstract

Effective biomedical information retrieval requires modeling domain semantics and hierarchical relationships among biomedical texts. Existing biomedical generative retrievers build on coarse binary relevance signals, limiting their ability to capture semantic overlap. We propose BioHiCL (Biomedical Retrieval with Hierarchical Multi-Label Contrastive Learning), which leverages hierarchical MeSH annotations to provide structured supervision for multi-label contrastive learning. Our models, BioHiCL-Base (0.1B) and BioHiCL-Large (0.3B), achieve promising performance on biomedical retrieval, sentence similarity, and question answering tasks, while remaining computationally efficient for deployment.

Citations

• BiCA: Effective Biomedical Dense Retrieval with Citation-Aware Hard Negatives
• Heterogeneous Contrastive Learning for Foundation Models and Beyond
• SPLADE-v3: New baselines for SPLADE
• C-Pack: Packed Resources For General Chinese Embeddings
• Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Training
• Text Embeddings by Weakly-Supervised Contrastive Pre-training
• SciFact-Open: Towards open-domain scientific claim verification
• SGPT: GPT Sentence Embeddings for Semantic Search
• Large Dual Encoders Are Generalizable Retrievers
• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
• SPECTER: Document-level Representation Learning using Citation-informed Transformers
• PubMedQA: A Dataset for Biomedical Research Question Answering
• A Fast Deep Learning Model for Textual Relevance in Biomedical Information Retrieval

Related

• Biomedical Entity Linking with Contrastive Context Matching
• Improving Biomedical Entity Linking with Retrieval-enhanced Learning
• MMRAG: Multi-Mode Retrieval-Augmented Generation with Large Language Models for Biomedical In-Context Learning
• BiomedRAG: A Retrieval Augmented Large Language Model for Biomedicine
• BioLORD-2023: semantic textual representations fusing large language models and clinical knowledge graph insights