Improving Code Understanding in Large Language Models through Concept-Aware Consistency Learning

2025/08/18 by Xiaoning Ren, Ren, Xiaoning, Qiang Hu +15
Computer Science · #Software Engineering Research #Online Learning and Analytics #Software Testing and Debugging Techniques

paper · pdf · doi:10.48550/arxiv.2508.12620

Abstract

Large language models (LLMs) have recently shown impressive results on diverse code-related tasks, benefiting from large-scale training and instruction tuning. However, studies reveal that their grasp of fundamental programming concepts, such as data flow and control flow, remains shallow, leading to fragile performance when code requires deeper reasoning. This limitation restricts the practical adoption of LLMs in real-world software development. To address this issue, this work introduces a counterfactual code augmentation framework combined with concept-aware tuning, designed to guide LLMs toward stronger conceptual understanding. Comprehensive evaluation across multiple models and benchmarks demonstrates the effectiveness of the proposed approach.

Related

• Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicates
• CodeSimpleQA: Scaling Factuality in Code Large Language Models
• On Code-Induced Reasoning in LLMs
• Coding Triangle: How Does Large Language Model Understand Code?
• Towards Concept-Aware Large Language Models