Detecting LLM-Generated Text with Performance Guarantees
2026/01/10 by Hongyi Zhou, Jin Zhu, Ying Yang +1 · 1 voice
Computer Science · Mathematics · #cs.CL #cs.LG #stat.AP #stat.ML
paper · pdf · doi:10.48550/arxiv.2601.06586
Abstract
Large language models (LLMs) such as GPT, Claude, Gemini, and Grok have been deeply integrated into our daily life. They now support a wide range of tasks -- from dialogue and email drafting to assisting with teaching and coding, serving as search engines, and much more. However, their ability to produce highly human-like text raises serious concerns, including the spread of fake news, the generation of misleading governmental reports, and academic misconduct. To address this practical problem, we train a classifier to determine whether a piece of text is authored by an LLM or a human. Our detector is deployed on an online CPU-based platform https://huggingface.co/spaces/stats-powered-ai/StatDetectLLM, and contains three novelties over existing detectors: (i) it does not rely on auxiliary information, such as watermarks or knowledge of the specific LLM used to generate the text; (ii) it more effectively distinguishes between human- and LLM-authored text; and (iii) it enables statistical inference, which is largely absent in the current literature. Empirically, our classifier achieves higher classification accuracy compared to existing detectors, while maintaining type-I error control, high statistical power, and computational efficiency.
Citations
- AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Attributing authorship via the perplexity of authorial language models
- Qwen3 Technical Report
- Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning
- Imitate Before Detect: Aligning Machine Stylistic Preference for Machine-Revised Text Detection
- Robust Detection of Watermarks for Large Language Models Under Human Edits
- Debiasing Watermarks for Large Language Models via Maximal Coupling
- GPT-4o System Card
- MedCalc-Bench: Evaluating Large Language Models for Medical Calculations
- Delving into LLM-assisted writing in biomedical publications through excess vocabulary
- ReMoDetect: Reward Models Recognize Aligned LLM's Generations
- RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors
- The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates
- A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rules
- Detecting Machine-Generated Texts by Multi-Population Aware Optimization for Maximum Mean Discrepancy
- Raidar: geneRative AI Detection viA Rewriting
- Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text
- A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
- Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature
- Large Language Models for Software Engineering: A Systematic Literature Review
- Large Language Models for Software Engineering: A Systematic Literature Review
- RADAR: Robust AI-Text Detection via Adversarial Learning
- Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts
- DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text
- DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text
- Students' Voices on Generative AI: Perceptions, Benefits, and Challenges in Higher Education
- ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text
- Cognitive Constraint Simulation and the Geometry of Human Authorship: A First-Principles Theory of AI Text Detection
- How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection
- Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods
- LexGLUE: A Benchmark Dataset for Legal Language Understanding in English
- MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer
- LoRA: Low-Rank Adaptation of Large Language Models
- Adversarially robust change point detection
- Efficient Attentions for Long Document Summarization
- Automatic Detection of Generated Text is Easiest when Humans are Fooled
- Release Strategies and the Social Impacts of Language Models
- GLTR: Statistical Detection and Visualization of Generated Text
- A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents
- Attention Is All You Need
- Character-level Convolutional Networks for Text Classification
Discussions
Related