On the Use of ArXiv as a Dataset

2019/04/30 by Colin B. Clement, Clement, Colin B., Matthew Bierbaum +6 · 23 citations
Computer Science · Decision Sciences · Physics and Astronomy · #Advanced Graph Neural Networks #FOS: Computer and information sciences #FOS: Physical sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Physics and Society (physics.soc-ph) #Scientific Computing and Data Management #Social and Information Networks (cs.SI) #Topic Modeling #cs.IR #cs.LG #cs.SI #physics.soc-ph

paper · pdf · doi:10.48550/arxiv.1905.00075

7 pages, 3 tables, 2 figures, ICLR 2019 workshop RLGM submission

arxiv created 2019/04/30 · openalex publication_date 2019/04/30 · arxiv updated 2019/05/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The arXiv has collected 1.5 million pre-print articles over 28 years, hosting literature from scientific fields including Physics, Mathematics, and Computer Science. Each pre-print features text, figures, authors, citations, categories, and other metadata. These rich, multi-modal features, combined with the natural graph structure---created by citation, affiliation, and co-authorship---makes the arXiv an exciting candidate for benchmarking next-generation models. Here we take the first necessary steps toward this goal, by providing a pipeline which standardizes and simplifies access to the arXiv's publicly available data. We use this pipeline to extract and analyze a 6.7 million edge citation graph, with an 11 billion word corpus of full-text research articles. We present some baseline classification results, and motivate application of more exciting generative graph models.

Citations

• ImageNet Large Scale Visual Recognition Challenge
• Inductive Representation Learning on Large Graphs
• Universal Sentence Encoder

Cited by

• Debugging Tabular Log as Dynamic Graphs
• iSeal: Encrypted Fingerprinting for Reliable LLM Ownership Verification
• SciNUP: Natural Language User Interest Profiles for Scientific Literature Recommendation
• Hidden Secrets in the arXiv: Discovering, Analyzing, and Preventing Unintentional Information Disclosure in Source Files of Scientific Preprints
• Learning What to Remember: Adaptive Probabilistic Memory Retention for Memory-Efficient Language Models
• Mid-Training of Large Language Models: A Survey
• MixtureVitae: Open Web-Scale Pretraining Dataset With High Quality Instruction and Reasoning Data Built from Permissive-First Text Sources
• Geometric Metrics and LLMs: What They Measure and When They Work
• SpeedCP: Fast Kernel-based Conditional Conformal Prediction
• Temporal Generalization: A Reality Check
• Beyond Easy Wins: A Text Hardness-Aware Benchmark for LLM-generated Text Detection
• Optimal Empirical Risk Minimization under Temporal Distribution Shifts
• On-the-Fly Adaptive Distillation of Transformer to Dual-State Linear Attention
• Large Language Models Encode Semantics and Alignment in Linearly Separable Representations
• Exploiting Leaderboards for Large-Scale Distribution of Malicious Models
• Scaling Attention to Very Long Sequences in Linear Time with Wavelet-Enhanced Random Spectral Attention (WERSA)
• Societal AI Research Has Become Less Interdisciplinary
• Disentangling Similarity and Relatedness in Topic Models
• Spectra 1.1: Scaling Laws and Efficient Inference for Ternary Language Models
• Modular versus Hierarchical: A Structural Signature of Topic Popularity in Mathematical Research
• Hush! Protecting Secrets During Model Training: An Indistinguishability Approach
• Changepoint Analysis of Topic Proportions in Temporal Text Data
• Continuous Domain Generalization

Related

• The arXiv of the future will not look like the arXiv
• Citegeist: Automated Generation of Related Work Analysis on the arXiv Corpus
• Are arXiv submissions on Wednesday better cited? Introducing Big Data methods in undergraduate courses on scientific computing
• Popularity of arXiv.org within Computer Science
• ResearchArcade: Graph Interface for Academic Tasks