2025/11/28 by Timothy Legge, Aaron Wang, Legge, Timothy +16 · 1 citation
Physics and Astronomy · #Data Analysis #FOS: Physical sciences #High Energy Physics - Experiment (hep-ex) #High Energy Physics - Phenomenology (hep-ph) #High-Energy Particle Collisions Research #Particle Detector Development and Performance #Particle physics theoretical and experimental studies #Statistics and Probability (physics.data-an)
paper · pdf · doi:10.48550/arxiv.2512.00210
openalex publication_date 2025/11/28 · openalex created_date 2025/12/03 · openalex updated_date 2026/07/28
Transformer-based models have achieved state-of-the-art performance in jet tagging at the CERN Large Hadron Collider (LHC), with the Particle Transformer (ParT) representing a leading example of such models. A striking feature of ParT is its sparse, nearly binary, attention structure, raising questions about the origin of this behavior and whether it encodes physically meaningful correlations. In this work, we investigate the source of ParT's sparse attention by comparing models trained on multiple benchmark datasets and examine the relative contributions of the attention term and the physics-inspired interaction matrix before softmax. We find that binary sparsity arises primarily from the attention mechanism itself, with the interaction matrix playing a secondary role. Moreove, we show that ParT is able to identify key jet substructure elements, such as leptons in semileptonic top decays, even without explicit particle identification inputs. These results provide new insight into the interpretability of transformer-based jet taggers and clarify the conditions under which sparse attention patterns emerge in ParT.