2024/10/15 by Sachita Nishal, Eric Lee, Nishal, Sachita +3 · 2 citations
Decision Sciences · #Computation and Language (cs.CL) #Computers and Society (cs.CY) #FOS: Computer and information sciences #H.4 #H.5 #Human-Computer Interaction (cs.HC) #Scientific Computing and Data Management
paper · pdf · doi:10.48550/arxiv.2410.12069
openalex publication_date 2024/10/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This study offers an initial evaluation of a human-in-the-loop system leveraging GPT-4 (a large language model or LLM), and Retrieval-Augmented Generation (RAG) to identify and define jargon terms in scientific abstracts, based on readers' self-reported knowledge. The system achieves fairly high recall in identifying jargon and preserves relative differences in readers' jargon identification, suggesting personalization as a feasible use-case for LLMs to support sense-making of complex information. Surprisingly, using only abstracts for context to generate definitions yields slightly more accurate and higher quality definitions than using RAG-based context from the fulltext of an article. The findings highlight the potential of generative AI for assisting science reporters, and can inform future work on developing tools to simplify dense documents.