2024/01/22 by Abhimanyu Hans, Avi Schwarzschild, Hans, Abhimanyu +13 · 2 voices · 94 citations
Chemistry · Computer Science · Engineering · #Alphabet #Artificial intelligence #Chemistry #Computer science #Engineering #Linguistics #Natural Language Processing Techniques #Natural language processing #Philosophy #Range (aeronautics) #Shot (pellet) #Spotting #Text Readability and Simplification #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2401.12070
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2024/01/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Detecting text generated by modern large language models is thought to be hard, as both LLMs and humans can exhibit a wide range of complex behaviors. However, we find that a score based on contrasting two closely related language models is highly accurate at separating human-generated and machine-generated text. Based on this mechanism, we propose a novel LLM detector that only requires simple calculations using a pair of pre-trained LLMs. The method, called Binoculars, achieves state-of-the-art accuracy without any training data. It is capable of spotting machine text from a range of modern LLMs without any model-specific modifications. We comprehensively evaluate Binoculars on a number of text sources and in varied situations. Over a wide range of document types, Binoculars detects over 90% of generated samples from ChatGPT (and other LLMs) at a false positive rate of 0.01%, despite not being trained on any ChatGPT data.