Exploring the Power of Diffusion Large Language Models for Software Engineering: An Empirical Investigation

2025/10/06 by Jingyao Zhang, Zhang, Jingyao, Tianlin Li +7 · 1 citation
Computer Science · #Advanced Malware Detection Techniques #FOS: Computer and information sciences #Software Engineering (cs.SE) #Software Engineering Research #Software Testing and Debugging Techniques

paper · pdf · doi:10.48550/arxiv.2510.04605

openalex publication_date 2025/10/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Autoregressive Large Language Models (AR-LLMs) are widely used in software engineering (SE) but face limitations in processing code structure information and suffer from high inference latency. Diffusion LLMs (DLLMs) offer a promising alternative with global bidirectional encoding and decoupled generation steps. This work presents the first comprehensive evaluation of DLLMs across the software development lifecycle, including code generation, defect detection, and program repair. On a large-scale benchmark of 52,937 tasks, 7Bparameter DLLMs outperform AR-LLMs with a 30% average accuracy improvement achieving a 113% gain on cross-file repair, while maintaining superior efficiency and reduced latency. Our results establish DLLMs as a superior paradigm for SE tasks.

Citations

• Persistent memory for AI coding agents: a pre-registered SWE-bench Verified benchmark
• Code Llama: Open Foundation Models for Code
• CodeT5+: Open Code Large Language Models for Code Understanding and Generation
• Diffusion-LM Improves Controllable Text Generation
• Step-unrolled Denoising Autoencoders for Text Generation
• Evaluating Large Language Models Trained on Code
• Language Models are Few-Shot Learners
• Generative Modeling by Estimating Gradients of the Data Distribution
• Non-Autoregressive Neural Machine Translation
• Attention Is All You Need

Cited by

• Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbone

Related

• Augmenting the Generality and Performance of Large Language Models for Software Engineering
• Large Language Models for Software Engineering: A Systematic Literature Review
• The Current Challenges of Software Engineering in the Era of Large Language Models
• Large Language Models for Software Engineering: A Systematic Literature Review
• Large Language Models for Software Engineering: Survey and Open Problems