2026/07/18 by Hyunjun Shin, Jiseung Jang, Jaewoo Maeng +1
#cs.DC
AI-assisted GPU programming is often framed as a kernel-generation loop: ask a model to produce faster CUDA code, benchmark the result, and repeat. This case study argues that contest-grade optimization involves more than improving the kernel body. We examine the Agent-Assisted submission by our team, MSInfer, to the MLSys 2026 FlashInfer Contest. The submission optimized Gated DeltaNet decode and prefill on NVIDIA B200/Blackwell and achieved an official 1.58× speedup, with approximate average latencies of 9.315 μs for decode and 239.48 μs for prefill. Our experience shows that even effective local kernel improvements can plateau when a workload requires structural reformulation and evaluator-aligned measurement. We therefore characterize AI-assisted kernel optimization as an end-to-end systems problem that encompasses algorithm design, workload specialization, measurement tooling, build and evaluation surfaces, evaluator alignment, and human interpretation.