Event Tokenization and Next-Token Prediction for Anomaly Detection at the Large Hadron Collider

2025/09/30 by Visive, Ambre, Moskvitina, Polina, Nellist, Clara +2 · 1 citation
#FOS: Physical sciences #High Energy Physics - Experiment (hep-ex) #High Energy Physics - Phenomenology (hep-ph)

paper · doi:10.48550/arxiv.2509.26218

Abstract

We propose a novel use of Large Language Models (LLMs) as unsupervised anomaly detectors in particle physics. Using lightweight LLM-like networks with encoder-based architectures trained to reconstruct background events via masked-token prediction, our method identifies anomalies through deviations in reconstruction performance, without prior knowledge of signal characteristics. Applied to searches for simultaneous four-top-quark production, this token-based approach shows competitive performance against established unsupervised methods and effectively captures subtle discrepancies in collider data, suggesting a promising direction for model-independent searches for new physics.

Citations

• ADALog: Adaptive Unsupervised Anomaly detection in Logs with Self-attention Masked Language Model
• Research on Cloud Platform Network Traffic Monitoring and Anomaly Detection System based on Large Language Models
• Universal Anomaly Detection at the LHC: Transforming Optimal Classifiers and the DDD Method
• Attention to the strengths of physical interactions: Transformer and graph-based event classification for particle physics experiments
• LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model
• On the Dangers of Stochastic Parrots
• Attention Is All You Need
• Adam: A Method for Stochastic Optimization

Cited by

• Reusable theory representations for colliders: a demonstrator SMEFT foundation model

Related

• New Physics Searches at the LHC through Event-based Anomaly Detection and Development of ADFilter Web-tool
• Universal Anomaly Detection at the LHC: Transforming Optimal Classifiers and the DDD Method
• Adversarially Learned Anomaly Detection on CMS Open Data: re-discovering the top quark
• Exploring Optimal Transport for Event-Level Anomaly Detection at the Large Hadron Collider
• Unsupervised Machine Learning for Anomaly Detection in LHC Collider Searches