Less Context, Same Performance: A RAG Framework for Resource-Efficient LLM-Based Clinical NLP

2025/05/23 by Satya Narayana Cheetirala, Cheetirala, Satya Narayana, Ganesh Raut +21 · 1 citation
Computer Science · Biochemistry, Genetics and Molecular Biology · #Natural Language Processing Techniques #Biomedical Text Mining and Ontologies

paper · pdf · doi:10.48550/arxiv.2505.20320

Abstract

Long text classification is challenging for Large Language Models (LLMs) due to token limits and high computational costs. This study explores whether a Retrieval Augmented Generation (RAG) approach using only the most relevant text segments can match the performance of processing entire clinical notes with large context LLMs. We begin by splitting clinical documents into smaller chunks, converting them into vector embeddings, and storing these in a FAISS index. We then retrieve the top 4,000 words most pertinent to the classification query and feed these consolidated segments into an LLM. We evaluated three LLMs (GPT4o, LLaMA, and Mistral) on a surgical complication identification task. Metrics such as AUC ROC, precision, recall, and F1 showed no statistically significant differences between the RAG based approach and whole-text processing (p > 0.05p > 0.05). These findings indicate that RAG can significantly reduce token usage without sacrificing classification accuracy, providing a scalable and cost effective solution for analyzing lengthy clinical documents.

Citations

• Large Language Models for Healthcare Text Classification: A Systematic Review
• Is Semantic Chunking Worth the Computational Cost?
• A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
• Almanac: Retrieval-Augmented Language Models for Clinical Medicine
• FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
• Big Bird: Transformers for Longer Sequences
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Longformer: The Long-Document Transformer
• Clinical Concept Embeddings Learned from Massive Sources of Multimodal\n Medical Data
• A Comparison of Word Embeddings for the Biomedical Natural Language Processing
• Improving average ranking precision in user searches for biomedical\n research datasets
• Billion-scale similarity search with GPUs

Cited by

• Patho-AgenticRAG: Towards Multimodal Agentic Retrieval-Augmented Generation for Pathology VLMs via Reinforcement Learning

Related

• CLI-RAG: A Retrieval-Augmented Framework for Clinically Structured and Context Aware Text Generation with LLMs
• Development and Testing of Retrieval Augmented Generation in Large Language Models -- A Case Study Report
• Large Language Models are Few-Shot Clinical Information Extractors
• Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs
• Information Extraction from Clinical Notes: Are We Ready to Switch to Large Language Models?