2025/12/21 by Minh Vu, Minh N. Vu, Konstantinos Slavakis +2 · 1 citation
Computer Science · #Advanced Multi-Objective Optimization Algorithms #Benchmark (surveying) #Estimator #Gaussian #Gaussian Processes and Bayesian Inference #Gaussian process #Mixing (physics) #Parametric statistics #Range (aeronautics) #Reinforcement Learning in Robotics #Reinforcement learning
paper · pdf · open access · doi:10.48550/arxiv.2512.18763
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2025/12/21 · openalex created_date 2025/12/24 · openalex updated_date 2026/07/28
Unlike their conventional use as estimators of probability density functions in reinforcement learning (RL), this paper introduces a novel function-approximation role for Gaussian mixture models (GMMs) as direct surrogates for Q-function losses. The proposed estimators, termed GMM-QFs, possess substantial representational capacity, as they are shown to be universal approximators over a broad class of functions. They are further embedded within Bellman residuals, where their learnable parameters--a fixed number of mixing weights, together with Gaussian mean vectors and covariance matrices--are inferred from data via optimization on a product Riemannian manifold. This geometric perspective on the parameter space naturally introduces Riemannian optimization into the policy-evaluation step of standard policy-iteration (PI) frameworks. Moreover, rigorous theoretical analysis establishes performance bounds on Q-function estimation error under the proposed PI scheme. Supporting numerical tests show that GMM-QFs deliver competitive performance and, in some cases, outperform state-of-the-art approaches across a range of benchmark RL tasks, all while maintaining a significantly smaller computational footprint than deep-learning methods.