LLM, Reporting In! Medical Information Extraction Across Prompting, Fine-tuning and Post-correction

2025/10/03 by Belmadani, Ikram, Hashemi, Parisa Nazari, Sebbag, Thomas +5
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR)

paper · doi:10.48550/arxiv.2510.03577

Abstract

This work presents our participation in the EvalLLM 2025 challenge on biomedical Named Entity Recognition (NER) and health event extraction in French (few-shot setting). For NER, we propose three approaches combining large language models (LLMs), annotation guidelines, synthetic data, and post-processing: (1) in-context learning (ICL) with GPT-4.1, incorporating automatic selection of 10 examples and a summary of the annotation guidelines into the prompt, (2) the universal NER system GLiNER, fine-tuned on a synthetic corpus and then verified by an LLM in post-processing, and (3) the open LLM LLaMA-3.1-8B-Instruct, fine-tuned on the same synthetic corpus. Event extraction uses the same ICL strategy with GPT-4.1, reusing the guideline summary in the prompt. Results show GPT-4.1 leads with a macro-F1 of 61.53% for NER and 15.02% for event extraction, highlighting the importance of well-crafted prompting to maximize performance in very low-resource scenarios.

Citations

• The Llama 3 Herd of Models
• LoRA: Low-Rank Adaptation of Large Language Models

Related

• FRASIMED: a Clinical French Annotated Resource Produced through Crosslingual BERT-Based Annotation Projection
• Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model prompting
• LLMs are not Zero-Shot Reasoners for Biomedical Information Extraction
• LLMs in Biomedicine: A study on clinical Named Entity Recognition
• Pre-trained Language Models and Few-shot Learning for Medical Entity Extraction