Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts

2023/06/07 by Eduard Tulchinskii, Kristian Kuznetsov, Tulchinskii, Eduard +13 · 21 citations
Computer Science · Social Sciences · #68T50 #Algebraic Topology (math.AT) #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computational and Text Analysis Methods #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2306.04723

openalex publication_date 2023/06/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

Rapidly increasing quality of AI-generated content makes it difficult to distinguish between human and AI-generated texts, which may lead to undesirable consequences for society. Therefore, it becomes increasingly important to study the properties of human texts that are invariant over different text domains and varying proficiency of human writers, can be easily calculated for any language, and can robustly separate natural and AI-generated texts regardless of the generation model and sampling method. In this work, we propose such an invariant for human-written texts, namely the intrinsic dimensionality of the manifold underlying the set of embeddings for a given text sample. We show that the average intrinsic dimensionality of fluent texts in a natural language is hovering around the value 9 for several alphabet-based languages and around 7 for Chinese, while the average intrinsic dimensionality of AI-generated texts for each language is ≈ 1.5 lower, with a clear statistical separation between human-generated and AI-generated distributions. This property allows us to build a score-based artificial text detector. The proposed detector's accuracy is stable over text domains, generator models, and human writer proficiency levels, outperforming SOTA detectors in model-agnostic and cross-domain scenarios by a significant margin.

Citations

• Intrinsic dimension of data representations in deep neural networks
• Release Strategies and the Social Impacts of Language Models
• Topology and geometry of data manifold in deep learning
• A Watermark for Large Language Models
• Cognitive Constraint Simulation and the Geometry of Human Authorship: A First-Principles Theory of AI Text Detection
• The geometry of hidden representations of large transformer models
• Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
• MGTBench: Benchmarking Machine-Generated Text Detection
• GPT detectors are biased against non-native English writers
• Exploiting Generative AI to Scale up Intelligent Tutoring Systems

Cited by

• Geometry of Decision Making in Language Models
• Unveiling Intrinsic Dimension of Texts: from Academic Abstract to Creative Story
• Measuring the Intrinsic Dimension of Earth Representations
• A Theoretical Analysis of Detecting Large Model-Generated Time Series
• Advancing Machine-Generated Text Detection from an Easy to Hard Supervision Perspective
• Machine Text Detectors are Membership Inference Attacks
• DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning
• When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection
• Memorization in Language Models through the Lens of Intrinsic Dimension
• Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection
• AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees
• Geometric Metrics and LLMs: What They Measure and When They Work
• LLMTrace: A Corpus for Classification and Fine-Grained Localization of AI-Written Text
• Improving Detection of Watermarked Language Models
• Authorship Attribution in Multilingual Machine-Generated Texts
• Droid: A Resource Suite for AI-Generated Code Detection
• On the Predictive Power of Representation Dispersion in Language Models
• On Convolutions, Intrinsic Dimension, and Diffusion Models
• Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddings
• AGENT-X: Adaptive Guideline-based Expert Network for Threshold-free AI-generated teXt detection
• LLM Output Detectability and Task Performance Can be Jointly Optimized
• Hitting a Moving Target: Test-Time Adaptation for AI Text Detection under Continual Distribution Shift
• Detecting LLM-Generated Text with Performance Guarantees
• OpenTuringBench: An Open-Model-based Benchmark and Framework for Machine-Generated Text Detection and Attribution
• Hallucination Detection in LLMs with Topological Divergence on Attention Graphs

Related

• On the Possibilities of AI-Generated Text Detection
• Robust and Fine-Grained Detection of AI Generated Texts
• Are AI-Generated Text Detectors Robust to Adversarial Perturbations?
• Are AI Detectors Good Enough? A Survey on Quality of Datasets With Machine-Generated Texts
• Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing