Integrating Performance Tools in Model Reasoning for GPU Kernel Optimization

2025/10/20 by Daniel Nichols, Konstantinos Parasyris, Nichols, Daniel +7
Computer Science · #Advanced Software Engineering Methodologies #Distributed #FOS: Computer and information sciences #Parallel #Parallel Computing and Optimization Techniques #Performance (cs.PF) #Software Engineering (cs.SE) #Software System Performance and Reliability #and Cluster Computing (cs.DC)

paper · pdf · doi:10.48550/arxiv.2510.17158

openalex publication_date 2025/10/20 · openalex created_date 2025/10/22 · openalex updated_date 2026/07/29

Abstract

Language models are now prevalent in software engineering with many developers using them to automate tasks and accelerate their development. While language models have been tremendous at accomplishing complex software engineering tasks, there are still many areas where they fail to deliver desirable results, for instance code performance related tasks. Tasks like optimization depend on many complex data from the environment, hardware, etc. that are not directly represented in source code. Recent efforts have seen large improvements in general code modeling tasks using chain-of-thought style reasoning, but these models still fail to comprehend how the environment interacts with code performance. In this paper we propose a methodology to train language models that can interact with performance tools during their reasoning process. We then demonstrate how this methodology can be used to train a state-of-the-art GPU kernel optimization model.

Citations

• SwizzlePerf: Hardware-Aware LLMs for GPU Kernel Performance Optimization
• GPU Kernel Scientist: An LLM-Driven Framework for Iterative Kernel Optimization
• SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents
• KernelBench: Can LLMs Write Efficient GPU Kernels?
• Language Models for Code Optimization: Survey, Challenges and Future Directions
• HPC-Coder-V2: Studying Code LLMs Across Low-Resource Parallel Languages
• EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
• Performance-Aligned LLMs for Generating Fast Code
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• PerfRL: A Small Language Model Framework for Efficient Code Optimization
• Magicoder: Empowering Code Generation with OSS-Instruct
• Persistent memory for AI coding agents: a pre-registered SWE-bench Verified benchmark
• Code Llama: Open Foundation Models for Code
• Instruction Tuning for Large Language Models: A Survey
• CodeT5+: Open Code Large Language Models for Code Understanding and Generation
• Self-Refine: Iterative Refinement with Self-Feedback
• Learning Performance-Improving Code Edits
• The Stack: 3 TB of permissively licensed source code
• Training language models to follow instructions with human feedback
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Training Verifiers to Solve Math Word Problems
• AxoNN: An asynchronous, message-driven parallel framework for extreme-scale deep learning
• Evaluating Large Language Models Trained on Code

Related

• PEAK: A Performance Engineering AI-Assistant for GPU Kernels Powered by Natural Language Transformations
• Can Large Language Models Predict Parallel Code Performance?
• Opal: A Modular Framework for Optimizing Performance using Analytics and LLMs
• Omniwise: Predicting GPU Kernels Performance with LLMs
• Tutoring LLM into a Better CUDA Optimizer