Keys in the Weights: Transformer Authentication Using Model-Bound Latent Representations

2025/11/02 by Okatan, Ayşe S., Akbaş, Mustafa İlhan, Kandel, Laxima Niure +1
#68M12 #68P25 #68Q32 #68Q87 #68T07 #94A17 #94A60 #Artificial Intelligence (cs.AI) #C.2.0 #C.2.2 #C.3 #Cryptography and Security (cs.CR) #D.4.6 #E.3 #FOS: Computer and information sciences #FOS: Electrical engineering #I.2.6 #I.5.1 #K.6.5 #Signal Processing (eess.SP) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2511.00973

Abstract

We introduce Model-Bound Latent Exchange (MoBLE), a decoder-binding property in Transformer autoencoders formalized as Zero-Shot Decoder Non-Transferability (ZSDN). In identity tasks using iso-architectural models trained on identical data but differing in seeds, self-decoding achieves more than 0.91 exact match and 0.98 token accuracy, while zero-shot cross-decoding collapses to chance without exact matches. This separation arises without injected secrets or adversarial training, and is corroborated by weight-space distances and attention-divergence diagnostics. We interpret ZSDN as model binding, a latent-based authentication and access-control mechanism, even when the architecture and training recipe are public: encoder's hidden state representation deterministically reveals the plaintext, yet only the correctly keyed decoder reproduces it in zero-shot. We formally define ZSDN, a decoder-binding advantage metric, and outline deployment considerations for secure artificial intelligence (AI) pipelines. Finally, we discuss learnability risks (e.g., adapter alignment) and outline mitigations. MoBLE offers a lightweight, accelerator-friendly approach to secure AI deployment in safety-critical domains, including aviation and cyber-physical systems.

Citations

• Private Transformer Inference in MLaaS: A Survey
• Block-Wise Encryption for Reliable Vision Transformer models
• On the Reliability of Watermarks for Large Language Models
• Model Stitching: Looking For Functional Similarity Between Representations
• Stitchable Neural Networks
• A Watermark for Large Language Models
• Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
• The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks
• LoRA: Low-Rank Adaptation of Large Language Models
• Revisiting Model Stitching to Compare Neural Representations
• The Power of Scale for Parameter-Efficient Prompt Tuning
• Similarity of Neural Network Representations Revisited
• Parameter-Efficient Transfer Learning for NLP
• Knockoff Nets: Stealing Functionality of Black-Box Models
• Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs
• Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring
• Attention Is All You Need
• An Introduction to Deep Learning for the Physical Layer
• Learning to Protect Communications with Adversarial Neural Cryptography
• Stealing Machine Learning Models via Prediction APIs
• Layer Normalization
• Convergent Learning: Do different neural networks learn the same representations?
• On the difficulty of training Recurrent Neural Networks
• A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
• On Information and Sufficiency
• Probabilistic encryption

Related

• Securing Transformer-based AI Execution via Unified TEEs and Crypto-protected Accelerators
• Unelicitable Backdoors in Language Models via Cryptographic Transformer Circuits
• SentinelLMs: Encrypted Input Adaptation and Fine-tuning of Language Models for Private and Secure Inference
• Can AI Keep a Secret? Contextual Integrity Verification: A Provable Security Architecture for LLMs
• I can't see it but I can Fine-tune it: On Encrypted Fine-tuning of Transformers using Fully Homomorphic Encryption