Quixer: A Quantum Transformer Model

2024/06/06 by Nikhil Khatri, Khatri, Nikhil, Gabriel Matos +5 · 7 citations
Computer Science · Engineering · Physics and Astronomy · #Advanced Electrical Measurement Techniques #FOS: Physical sciences #Quantum Computing Algorithms and Architecture #Quantum Physics (quant-ph) #Spectroscopy and Quantum Chemical Studies

paper · pdf · doi:10.48550/arxiv.2406.04305

openalex publication_date 2024/06/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Progress in the realisation of reliable large-scale quantum computers has motivated research into the design of quantum machine learning models. We present Quixer: a novel quantum transformer model which utilises the Linear Combination of Unitaries and Quantum Singular Value Transform primitives as building blocks. Quixer operates by preparing a superposition of tokens and applying a trainable non-linear transformation to this mix. We present the first results for a quantum transformer model applied to a practical language modelling task, obtaining results competitive with an equivalent classical baseline. In addition, we include resource estimates for evaluating the model on quantum hardware, and provide an open-source implementation for classical simulation. We conclude by highlighting the generality of Quixer, showing that its parameterised components can be substituted with fixed structures to yield new classes of quantum transformers.

Cited by

• Practical Hybrid Quantum Language Models with Observable Readout on Real Hardware
• Geometric Optimization on Lie Groups: A Lie-Theoretic Explanation of Barren Plateau Mitigation for Variational Quantum Algorithms
• Hybrid Quantum-Classical Selective State Space Artificial Intelligence
• You Only Measure Once: On Designing Single-Shot Quantum Machine Learning Models
• Quantum Graph Attention Network: A Novel Quantum Multi-Head Attention Mechanism for Graph Learning
• Resting-state fMRI Analysis using Quantum Time-series Transformer
• PGKET: A Photonic Gaussian Kernel Enhanced Transformer

Related

• Quantum Transformer: Accelerating model inference via quantum linear algebra
• A Survey of Quantum Transformers: Architectures, Challenges and Outlooks
• Quantum Embedding with Transformer for High-dimensional Data
• Transpiling quantum circuits by a transformers-based algorithm
• GPT on a Quantum Computer