CARPO: Leveraging Listwise Learning-to-Rank for Context-Aware Query Plan Optimization

2025/09/03 by Wenrui Zhou, Zhou, Wenrui, Qiyu Liu +7 · 1 voice
Computer Science · #Advanced Database Systems and Queries #Cloud Computing and Resource Management #Data Management and Algorithms #Databases (cs.DB) #FOS: Computer and information sciences #cs.DB

paper · pdf · doi:10.48550/arxiv.2509.03102

openalex publication_date 2025/09/03 · arxiv published 2025/09/03 · openalex created_date 2025/10/10 · arxiv updated 2025/10/21 · openalex updated_date 2026/07/28

Abstract

Efficient data processing is increasingly vital, with query optimizers playing a fundamental role in translating SQL queries into optimal execution plans. Traditional cost-based optimizers, however, often generate suboptimal plans due to flawed heuristics and inaccurate cost models, leading to the emergence of Learned Query Optimizers (LQOs). To address challenges in existing LQOs, such as the inconsistency and suboptimality inherent in pairwise ranking methods, we introduce CARPO, a generic framework leveraging listwise learning-to-rank for context-aware query plan optimization. CARPO distinctively employs a Transformer-based model for holistic evaluation of candidate plan sets and integrates a robust hybrid decision mechanism, featuring Out-Of-Distribution (OOD) detection with a top-k fallback strategy to ensure reliability. Furthermore, CARPO can be seamlessly integrated with existing plan embedding techniques, demonstrating strong adaptability. Comprehensive experiments on TPC-H and STATS benchmarks demonstrate that CARPO significantly outperforms both native PostgreSQL and Lero, achieving a Top-1 Rate of 74.54% on the TPC-H benchmark compared to Lero's 3.63%, and reducing the total execution time to 3719.16 ms compared to PostgreSQL's 22577.87 ms.

Citations

• Balsa: Learning a Query Optimizer Without Expert Demonstrations
• An End-to-End Learning-based Cost Estimator
• Learned Cardinalities: Estimating Correlated Joins with Deep Learning
• Learning to Optimize Join Queries With Deep Reinforcement Learning
• Learning State Representations for Query Optimization with Deep Reinforcement Learning
• Attention Is All You Need
• Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Discussions

• #cs.DB Origin | Interest | Match [bsky, 0 points, 0 comments]

Related

• Lero: A Learning-to-Rank Query Optimizer
• LIMAO: A Framework for Lifelong Modular Learned Query Optimization
• COOOL: A Learning-To-Rank Approach for SQL Hint Recommendations
• JOB-Complex: A Challenging Benchmark for Traditional & Learned Query Optimization
• Guided Automated Learning for query workload re-Optimization