Quantifying Cross-Attention Interaction in Transformers for Interpreting TCR-pMHC Binding

2025/07/03 by Jiarui Li, Li, Jiarui, Yin, Zixiang +8 · 2 citations
Medicine · #Biomolecules (q-bio.BM) #Computational Engineering #FOS: Biological sciences #FOS: Computer and information sciences #Finance #Machine Learning (cs.LG) #Medical Imaging Techniques and Applications #and Science (cs.CE)

paper · pdf · doi:10.48550/arxiv.2507.03197

openalex publication_date 2025/07/03 · openalex created_date 2025/10/20 · openalex updated_date 2026/07/28

Abstract

CD8+ "killer" T cells and CD4+ "helper" T cells play a central role in the adaptive immune system by recognizing antigens presented by Major Histocompatibility Complex (pMHC) molecules via T Cell Receptors (TCRs). Modeling binding between T cells and the pMHC complex is fundamental to understanding basic mechanisms of human immune response as well as in developing therapies. While transformer-based models such as TULIP have achieved impressive performance in this domain, their black-box nature precludes interpretability and thus limits a deeper mechanistic understanding of T cell response. Most existing post-hoc explainable AI (XAI) methods are confined to encoder-only, co-attention, or model-specific architectures and cannot handle encoder-decoder transformers used in TCR-pMHC modeling. To address this gap, we propose Quantifying Cross-Attention Interaction (QCAI), a new post-hoc method designed to interpret the cross-attention mechanisms in transformer decoders. Quantitative evaluation is a challenge for XAI methods; we have compiled TCR-XAI, a benchmark consisting of 274 experimentally determined TCR-pMHC structures to serve as ground truth for binding. Using these structures we compute physical distances between relevant amino acid residues in the TCR-pMHC interaction region and evaluate how well our method and others estimate the importance of residues in this region across the dataset. We show that QCAI achieves state-of-the-art performance on both interpretability and prediction accuracy under the TCR-XAI benchmark.

Citations

• Sequence-Only Prediction of Binding Affinity Changes: A Robust and Interpretable Model for Antibody Engineering
• Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformer
• AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers
• Personalized RNA neoantigen vaccines stimulate T cells in pancreatic cancer
• Can we predict T cell specificity with digital biology and machine learning?
• Disentangled Wasserstein Autoencoder for T-Cell Receptor Engineering
• XAI for Transformers: Better Explanations through Conservative Propagation
• Generic Attention-model Explainability for Interpreting Bi-Modal and\n Encoder-Decoder Transformers
• Learning Transferable Visual Models From Natural Language Supervision
• Quantifying Attention Flow in Transformers
• Attention is not not Explanation
• Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned
• Attention Is All You Need
• Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
• Layer-wise Relevance Propagation for Neural Networks with Local\n Renormalization Layers
• Learning Deep Features for Discriminative Localization
• Microsoft COCO: Common Objects in Context
• NetMHCpan, a method for MHC class I binding prediction beyond humans
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Cited by

• TCR-EML: Explainable Model Layers for TCR-pMHC Prediction
• Rational Multi-Modal Transformers for TCR-pMHC Prediction

Related

• Rational Multi-Modal Transformers for TCR-pMHC Prediction
• TCR-EML: Explainable Model Layers for TCR-pMHC Prediction
• Attention-aware contrastive learning for predicting T cell receptor-antigen binding specificity
• Deciphering small sequence differences in T cell receptor-antigen pairing
• A unified cross-attention model for predicting antigen binding specificity to both HLA and TCR molecules