Devstral: Fine-tuning Language Models for Coding Agent Applications

2025/08/08 by Rastogi, Abhinav, Yang, Adam, Jiang, Albert Q. +100 · 3 citations
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Software Engineering (cs.SE)

paper · doi:10.48550/arxiv.2509.25193

Abstract

We introduce Devstral-Small, a lightweight open source model for code agents with the best performance among models below 100B size. In this technical report, we give an overview of how we design and develop a model and craft specializations in agentic software development. The resulting model, Devstral-Small is a small 24B model, fast and easy to serve. Despite its size, Devstral-Small still attains competitive performance compared to models more than an order of magnitude larger.

Citations

• Training Software Engineering Agents and Verifiers with SWE-Gym
• Qwen2.5-Coder Technical Report
• SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
• Executable Code Actions Elicit Better LLM Agents
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• Code Llama: Open Foundation Models for Code
• Self-Refine: Iterative Refinement with Self-Feedback
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Program Synthesis with Large Language Models
• Evaluating Large Language Models Trained on Code

Cited by

• Agentic Software Issue Resolution with Large Language Models: A Survey
• On Randomness in Agentic Evals
• QAgent: An LLM-based Multi-Agent System for Autonomous OpenQASM programming

Related

• A Self-Improving Coding Agent
• MapCoder-Lite: Distilling Multi-Agent Coding into a Single Small LLM
• Small Language Models for Agentic Systems: A Survey of Architectures, Capabilities, and Deployment Trade offs
• Lita: Light Agent Uncovers the Agentic Coding Capabilities of LLMs
• Software Development Life Cycle Perspective: A Survey of Benchmarks for Code Large Language Models and Agents