In-depth Research Impact Summarization through Fine-Grained Temporal Citation Analysis

2025/05/20 by Hiba Arnaout, Arnaout, Hiba, Noy Sternlicht +5 · 1 voice · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Digital Libraries (cs.DL) #FOS: Computer and information sciences #cs.AI #cs.DL

paper · pdf · doi:10.48550/arxiv.2505.14838

arxiv published 2025/05/20 · arxiv updated 2026/04/16

Abstract

Understanding the impact of scientific publications is crucial for identifying breakthroughs and guiding future research. Traditional metrics based on citation counts often miss the nuanced ways a paper contributes to its field. In this work, we propose a new task: generating nuanced, expressive, and time-aware impact summaries that capture both praise (confirmation citations) and critique (correction citations) through the evolution of fine-grained citation intents. We introduce an evaluation framework tailored to this task, showing moderate to strong human correlation on subjective metrics such as insightfulness. Expert feedback from professors reveals a strong interest in these summaries and suggests future improvements. Data and code are made available.

Citations

• Qwen3 Technical Report
• OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
• Beyond Relevant Documents: A Knowledge-Intensive Approach for Query-Focused Summarization using Large Language Models
• Systematic Task Exploration with LLMs: A Study in Citation Text Generation
• From Local to Global: A Graph RAG Approach to Query-Focused Summarization
• FABLES: Evaluating faithfulness and content selection in book-length summarization
• PST-Bench: Tracing and Benchmarking the Source of Publications
• Forecasting high-impact research topics via machine learning on evolving knowledge graphs
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
• Lost in the Middle: How Language Models Use Long Contexts
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Large Language Models are not Fair Evaluators
• SciReviewGen: A Large-scale Dataset for Automatic Literature Review Generation
• G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
• In-Context Retrieval-Augmented Language Models
• Large Language Models Encode Clinical Knowledge
• Large language models encode clinical knowledge
• When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
• Towards Generating Citation Sentences for Multiple References with Intent Control
• Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
• MultiCite: Modeling realistic citations requires moving beyond the single-sentence single-label setting
• ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision
• Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks
• Natural language processing to identify the creation and impact of new technologies in patent text: Code, data, and new measures
• Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles
• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
• Citation Cascade and the Evolution of Topic Relevance
• Rumor Detection on Social Media with Bi-Directional Graph Convolutional Networks
• OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction
• Commonsense Knowledge Mining from Pretrained Models
• Structural Scaffolds for Citation Intent Classification in Scientific Publications
• Fake News Detection on Social Media using Geometric Deep Learning
• Antibiotic resistance in Pseudomonas aeruginosa: mechanisms and alternative therapeutic strategies
• Neural Open Information Extraction
• Attention Is All You Need
• Coupled Generative Adversarial Networks

Cited by

• Large Language Models for History, Philosophy, and Sociology of Science: Interpretive Uses, Methodological Challenges, and Critical Perspectives

Discussions

• We introduce a new task that tracks the evolving influence of a paper using 𝗳𝗶𝗻𝗲-𝗴𝗿𝗮𝗶𝗻𝗲𝗱 𝗰𝗶𝘁𝗮𝘁𝗶𝗼𝗻 𝗶𝗻𝘁𝗲𝗻𝘁𝘀 🧠⏳. 📄 Paper: arxiv.org/pdf/2505.14838 🔗 Project: ukplab.github.io [bsky, 0 points, 1 comments]

Related

• A Measure of Research Taste
• High-impact and transformative science (HITS) metrics: Definition, exemplification, and comparison
• Comprehensive Manuscript Assessment with Text Summarization Using 69707 articles
• Fusion of the Power from Citations: Enhance your Influence by Integrating Information from References
• A Fitness Model for Scholarly Impact Analysis