2025/05/13 by Hull, Gavin, Alex Bihlo, Bihlo, Alex
Decision Sciences · Computer Science · Medicine · #scientometrics and bibliometrics research #Topic Modeling #Artificial Intelligence in Healthcare and Education
paper · pdf · doi:10.48550/arxiv.2505.08941
Predicting the future citation rates of academic papers is an important step toward the automation of research evaluation and the acceleration of scientific progress. We present ForeCite, a simple but powerful framework to append pre-trained causal language models with a linear head for average monthly citation rate prediction. Adapting transformers for regression tasks, ForeCite achieves a test correlation of ρ= 0.826 on a curated dataset of 900K+ biomedical papers published between 2000 and 2024, a 27-point improvement over the previous state-of-the-art. Comprehensive scaling-law analysis reveals consistent gains across model sizes and data volumes, while temporal holdout experiments confirm practical robustness. Gradient-based saliency heatmaps suggest a potentially undue reliance on titles and abstract texts. These results establish a new state-of-the-art in forecasting the long-term influence of academic research and lay the groundwork for the automated, high-fidelity evaluation of scientific contributions.