QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generation

2025/05/30 by Yaoyu Zhu, Di Huang, Zhu, Yaoyu +35 · 1 voice · 8 citations
Computer Science · #FOS: Computer and information sciences #Hardware Architecture (cs.AR) #Machine Learning (cs.LG) #Natural Language Processing Techniques #Programming Languages (cs.PL) #Software Engineering Research #cs.AR #cs.LG #cs.PL

paper · pdf · doi:10.48550/arxiv.2505.24183

openalex publication_date 2025/05/30 · arxiv published 2025/05/30 · openalex created_date 2025/10/06 · arxiv updated 2026/02/23 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) trained via reinforcement learning with verifiable reward (RLVR) have achieved breakthroughs on tasks with explicit, automatable verification, such as software programming and mathematical problems. Extending RLVR to electronic design automation (EDA), especially automatically generating hardware description languages (HDLs) like Verilog from natural-language (NL) specifications, however, poses three key challenges: the lack of automated and accurate verification environments, the scarcity of high-quality NL-code pairs, and the prohibitive computation cost of RLVR. To this end, we introduce CodeV-R1, an RLVR framework for training Verilog generation LLMs. First, we develop a rule-based testbench generator that performs robust equivalence checking against golden references. Second, we propose a round-trip data synthesis method that pairs open-source Verilog snippets with LLM-generated NL descriptions, verifies code-NL-code consistency via the generated testbench, and filters out inequivalent examples to yield a high-quality dataset. Third, we employ a two-stage "distill-then-RL" training pipeline: distillation for the cold start of reasoning abilities, followed by adaptive DAPO, our novel RLVR algorithm that can reduce training cost by adaptively adjusting sampling rate. The resulting model, CodeV-R1-7B, achieves 68.6% and 72.9% pass@1 on VerilogEval v2 and RTLLM v1.1, respectively, surpassing prior state-of-the-art by 12~20%, while even exceeding the performance of 671B DeepSeek-R1 on RTLLM. We have released our model, training code, and dataset to facilitate research in EDA and LLM communities.

Citations

• Insights from Verification: Training a Verilog Generation LLM with Reinforcement Learning with Testbench Feedback
• ReasoningV: Efficient Verilog Code Generation with Adaptive Hybrid Reasoning Model
• Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model
• SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
• Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding
• LIMO: Less is More for Reasoning
• s1: Simple test-time scaling
• Kimi k1.5: Scaling Reinforcement Learning with LLMs
• Qwen2.5 Technical Report
• Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems
• O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?
• CorrectBench: Automatic Testbench Generation with Functional Self-Correction using LLMs for HDL Design
• OpenLLM-RTL: Open Dataset and Benchmark for LLM-Aided Design RTL Generation
• CraftRTL: High-quality Synthetic Data Generation for Verilog Code Models with Correct-by-Construction Non-Textual Representations and Targeted Code Repair
• OriGen:Enhancing RTL Code Generation with Code-to-Code Augmentation and Self-Reflection
• Large Language Model for Verilog Generation with Code-Structure-Guided Reinforcement Learning
• AutoVCoder: A Systematic Framework for Automated Verilog Code Generation using LLMs
• AutoBench: Automatic Testbench Generation and Evaluation Using LLMs for HDL Design
• MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation
• LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
• BetterV: Controlled Verilog Generation with Discriminative Guidance
• TACO: Topics in Algorithmic COde generation dataset
• VerilogEval: Evaluating Large Language Models for Verilog Code Generation
• One Embedder, Any Task: Instruction-Finetuned Text Embeddings
• Evaluating Large Language Models Trained on Code
• Measuring Coding Challenge Competence With APPS
• Revisiting VerilogEval: A Year of Improvements in Large-Language Models for Hardware Code Generation

Cited by

• Replay Failures as Successes: Sample-Efficient Reinforcement Learning for Instruction Following
• CircuitWeave: Topology-Behavior Alignment for Executable Multimodal RTL Generation
• Large Language Model for Verilog Code Generation: Literature Review and the Road Ahead
• Understanding and Mitigating Errors of LLM-Generated RTL Code
• StepFun-Formalizer: Unlocking the Autoformalization Potential of LLMs through Knowledge-Reasoning Fusion
• AnalogCoder-Pro: Unifying Analog Circuit Generation and Optimization via Multi-modal LLMs
• PRO-V-R1: Reasoning Enhanced Programming Agent for RTL Verification
• ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design

Discussions

• Curious to see how this would work given the challenges of reliable testbench generation (other than data scarcity--here of code x natural language pairs) arxiv.org/abs/2505.24183 [bsky, 2 points, 1 comments]

Related

• Insights from Verification: Training a Verilog Generation LLM with Reinforcement Learning with Testbench Feedback
• VeriReason: Reinforcement Learning with Testbench Feedback for Reasoning-Enhanced Verilog Generation
• VERIRL: Boosting the LLM-based Verilog Code Generation via Reinforcement Learning
• ReasoningV: Efficient Verilog Code Generation with Adaptive Hybrid Reasoning Model
• VRank: Enhancing Verilog Code Generation from Large Language Models via Self-Consistency