Policy Gradients using Variational Quantum Circuits

2022/03/20 by André Sequeira, Sequeira, André, Luís Paulo Santos +3 · 6 citations
Computer Science · Engineering · #Advancements in Semiconductor Devices and Circuit Design #FOS: Computer and information sciences #FOS: Physical sciences #Machine Learning (cs.LG) #Quantum Computing Algorithms and Architecture #Quantum Information and Cryptography #Quantum Physics (quant-ph)

paper · pdf · doi:10.48550/arxiv.2203.10591

openalex publication_date 2022/03/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Variational Quantum Circuits are being used as versatile Quantum Machine Learning models. Some empirical results exhibit an advantage in supervised and generative learning tasks. However, when applied to Reinforcement Learning, less is known. In this work, we considered a Variational Quantum Circuit composed of a low-depth hardware-efficient ansatz as the parameterized policy of a Reinforcement Learning agent. We show that an ε-approximation of the policy gradient can be obtained using a logarithmic number of samples concerning the total number of parameters. We empirically verify that such quantum models behave similarly or even outperform typical classical neural networks used in standard benchmarking environments and in quantum control, using only a fraction of the parameters. Moreover, we study the Barren Plateau phenomenon in quantum policy gradients using the Fisher Information Matrix spectrum.

Cited by

• Formal Verification of Noisy Quantum Reinforcement Learning Policies
• Dissecting Quantum Reinforcement Learning: A Systematic Evaluation of Key Components
• Quantum Reinforcement Learning: Recent Advances and Future Directions
• Scalable Quantum Reinforcement Learning on NISQ Devices with Dynamic-Circuit Qubit Reuse and Grover Optimization
• Gradients, parallelism, and variance of quantum estimates
• HCQA: Hybrid Classical-Quantum Agent for Generating Optimal Quantum Sensor Circuits

Related

• Variational Quantum Circuits for Deep Reinforcement Learning
• Breaking Through Barren Plateaus: Reinforcement Learning Initializations for Deep Variational Quantum Circuits
• Optimizing Quantum Variational Circuits with Deep Reinforcement Learning
• Trainability issues in quantum policy gradients
• A Study on Optimization Techniques for Variational Quantum Circuits in Reinforcement Learning