Reasoning as a Resource: Optimizing Fast and Slow Thinking in Code Generation Models

2025/06/11 by Zongjie Li, Shuai Wang, Li, Zongjie +1 · 2 citations
Computer Science · Decision Sciences · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Model-Driven Software Engineering Techniques #Scientific Computing and Data Management #Software Engineering (cs.SE) #Software Engineering Research

paper · pdf · doi:10.48550/arxiv.2506.09396

openalex publication_date 2025/06/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This position paper proposes a fundamental shift in designing code generation models: treating reasoning depth as a controllable resource. Rather than being an incidental byproduct of prompting, we argue that the trade-off between rapid, direct answers ("fast thinking") and elaborate, chain-of-thought deliberation ("slow thinking") must be explicitly managed. We contend that optimizing reasoning budgets across the entire model lifecycle - from synthetic data creation and benchmarking to real-world deploymen - can unlock superior trade-offs among accuracy, latency, and cost. This paper outlines how adaptive control over reasoning can enrich supervision signals, motivate new multi-dimensional benchmarks, and inform cost-aware, security-conscious deployment policies. By viewing fast and slow thinking as complementary modes to be scheduled, we envision coding agents that think deep when necessary and act fast when possible.

Citations

• Differentiation-Based Extraction of Proprietary Data from Fine-Tuned LLMs
• AdaptThink: Reasoning Models Can Learn When to Think
• RepoAudit: An Autonomous LLM-Agent for Repository-Level Code Auditing
• HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation
• API-guided Dataset Synthesis to Finetune Large Code Models
• SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner
• SPVR: syntax-to-prompt vulnerability repair based on large language models
• Exploring Multi-Lingual Bias of Large Code Models in Code Generation
• StarCoder 2 and The Stack v2: The Next Generation
• Refining Decompiled C Code with Large Language Models
• Split and Merge: Aligning Position Biases in LLM-based Evaluators
• Fixing Large Language Models' Specification Misunderstanding for Better Code Generation
• Code Llama: Open Foundation Models for Code
• Structured Chain-of-Thought Prompting for Code Generation
• Structured Chain-of-Thought Prompting for Code Generation
• StarCoder: may the source be with you!
• Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework
• Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes
• CCTEST: Testing and Repairing Code Completion Systems
• Unleashing the Power of Compiler Intermediate Representation to Enhance Neural Program Embeddings
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
• Program Synthesis with Large Language Models
• Evaluating Large Language Models Trained on Code
• Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models

Cited by

• Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learning
• Effective and Efficient Context Retrieval via Partial Dependency Graph for Repository-Level Code Generation

Related

• Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMs
• Efficient Reasoning Models: A Survey
• Reasoning Efficiently Through Adaptive Chain-of-Thought Compression: A Self-Optimizing Framework
• Z1: Efficient Test-time Scaling with Code
• Think in Blocks: Adaptive Reasoning from Direct Response to Deep Reasoning