Program Synthesis with Large Language Models

2021/08/16 by Jacob Austin, Augustus Odena, Austin, Jacob +21 · 1 voice · 609 citations
Computer Science · #Software Engineering Research #Software Testing and Debugging Techniques #Topic Modeling #cs.LG #cs.PL

paper · pdf · doi:10.48550/arxiv.2108.07732

openalex publication_date 2021/08/16 · openalex created_date 2021/08/30 · openalex updated_date 2026/07/28

Abstract

This paper explores the limits of the current generation of large language models for program synthesis in general purpose programming languages. We evaluate a collection of such models (with between 244M and 137B parameters) on two new benchmarks, MBPP and MathQA-Python, in both the few-shot and fine-tuning regimes. Our benchmarks are designed to measure the ability of these models to synthesize short Python programs from natural language descriptions. The Mostly Basic Programming Problems (MBPP) dataset contains 974 programming tasks, designed to be solvable by entry-level programmers. The MathQA-Python dataset, a Python version of the MathQA benchmark, contains 23914 problems that evaluate the ability of the models to synthesize code from more complex text. On both datasets, we find that synthesis performance scales log-linearly with model size. Our largest models, even without finetuning on a code dataset, can synthesize solutions to 59.6 percent of the problems from MBPP using few-shot learning with a well-designed prompt. Fine-tuning on a held-out portion of the dataset improves performance by about 10 percentage points across most model sizes. On the MathQA-Python dataset, the largest fine-tuned model achieves 83.8 percent accuracy. Going further, we study the model's ability to engage in dialog about code, incorporating human feedback to improve its solutions. We find that natural language feedback from a human halves the error rate compared to the model's initial prediction. Additionally, we conduct an error analysis to shed light on where these models fall short and what types of programs are most difficult to generate. Finally, we explore the semantic grounding of these models by fine-tuning them to predict the results of program execution. We find that even our best models are generally unable to predict the output of a program given a specific input.

Citations

• Learning to Execute
• GenProg: A Generic Method for Automatic Software Repair
• Semi-supervised Sequence Learning
• Neural GPUs Learn Algorithms
• Exploring the Limits of Language Modeling
• Latent Predictor Networks for Code Generation
• DeepCoder: Learning to Write Programs
• A Syntactic Neural Model for General-Purpose Code Generation
• A Convolutional Attention Network for Extreme Summarization of Source Code
• LambdaNet: Probabilistic Type Inference using Graph Neural Networks
• CodeBERT: A Pre-Trained Model for Programming and Natural Languages
• On the Dangers of Stochastic Parrots

Cited by

• Layer-wise LoRA fine-tuning: a similarity metric approach
• \kappa-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating
• Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Model
• Enough is as good as a feast: A Comprehensive Analysis of How Reinforcement Learning Mitigates Task Conflicts in LLMs
• MoE²-LoRA: When MoE Models Meet MoE-style Low-Rank Adaptation
• MineValiCoder: Reliable Code Generation with Test Case Quality Mining and Bipartite Graph-Based Mutual Validation
• Vibe Coding: An Experiment with Test-Driven Development
• Position: Quantum Program Generation Must Prioritize Validity Over Probabilistic Scaling
• FlowBot: Inducing LLM Workflows with Bilevel Optimization and Textual Gradients
• PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization
• Skillware: A Software Ontology and Engineering Lifecycle for Persistent Behavioral Artifacts
• Cross-Model LLM Code Review: Should you use Claude to review Codex or vice versa?
• LaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models
• REGEN: Replay-recycling for Expert-to-Generalist distillation with Offline Reinforcement Learning
• Spaghetti Architect: A Contamination-Resistant, By-Construction-Labelled, Multi-Language Code Dataset Generator
• TraceDev: A Traceability-Driven Multi-agent Framework for Requirement-to-Code Development
• CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning
• Program Synthesis for Simulation-Based Inference: Joint Model Selection and Parameter Estimation
• Heteroskedastic Signals in Budgeted LLM Verification: Structural Heterogeneity Limits Optimization Gains
• Beyond Resolved Rate: A Non-Functional Quality Study
• TopoTuner: Topological Finetuning of Large Language Models
• NIRVANA: Structured Pruning Reimagined for Large Language Model Compression
• Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering
• AgentBrew: Lifelong Knowledge Brewing from Strong Teachers to Weak LLM Agents
• Verified LLM-Driven Synthesis for Concept Design
• Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios
• Code-MUE: Measuring Code LLMs' Uncertainty through Execution-based Semantic Interaction Graphs
• Mask-Aware Policy Gradients for Diffusion Language Models
• Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?
• D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding
• ExecuGraph: A Multi-Agent, Execution-Grounded Framework for Reliable Backend Code Synthesis with Large Language Models
• Explaining Attention with Program Synthesis
• Coupled Hierarchical Search over Topology and Execution for Agentic Workflow Synthesis
• DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding
• FindStatBench: Evaluating Large Language Models on Combinatorial Code Synthesis
• Cross-Dialect Generalization Without Retraining: Benchmarks and Evaluation of Schema-Derived Constrained Decoding for MLIR
• Domyn-Small: A European 10B Reasoning Language Model
• Information-Theoretic Limits of Reliability and Scaling in Language Models
• Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs
• Automatically Evolving Prompt Guidelines for Task-Specific Optimization
• Eta Given Delta: Defining LLM Tool Efficiency With Marginal Tool Utility
• Embarrassingly Simple Self-Distillation Improves Code Generation
• Effective Distillation to Hybrid xLSTM Architectures
• Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights
• SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks
• SWE-CI: Evaluating Agent Capabilities in Maintaining Codebases via Continuous Integration
• Soft Contamination Means Benchmarks Test Shallow Generalization
• SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
• A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training
• Do Large Language Models Know What They Are Capable Of?
• Train for Truth, Keep the Skills: Binary Retrieval-Augmented Reward Mitigates Hallucinations
• LLaDA-MoE: A Sparse MoE Diffusion Language Model
• Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs
• Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search
• Fast and Simplex: 2-Simplicial Attention in Triton
• Text-to-LoRA: Instant Transformer Adaption
• Mercury: Ultra-Fast Language Models Based on Diffusion
• Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
• Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning
• Type-Constrained Code Generation with Language Models
• QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?
• Large Language Diffusion Models
• SWE-Lancer: Can Frontier LLMs Earn 1 Million from Real-World Freelance Software Engineering?
• Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
• Generating Computational Cognitive Models using Large Language Models
• A Probabilistic Framework for LLM-Based Model Discovery
• Post-Training Quantization of OpenPangu Models for Efficient Deployment on Atlas A2
• Anka: A Domain-Specific Language for Reliable LLM Code Generation
• BenCSSmark: Making the Social Sciences Count in LLM Research
• FasterPy: An LLM-based Code Execution Efficiency Optimization Framework
• WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference
• Rethinking the Capability of Fine-Tuned Language Models for Automated Vulnerability Repair
• M2G-Eval: Enhancing and Evaluating Multi-granularity Multilingual Code Generation
• Learning When Not to Attend Globally
• Efficient Multi-Model Orchestration for Self-Hosted Large Language Models
• Beyond Single Bugs: Benchmarking Large Language Models for Multi-Vulnerability Detection
• Shape of Thought: When Distribution Matters More than Correctness in Reasoning Tasks
• State-dependent error correlations shape voting thresholds in committees of AI agents
• DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention
• Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion
• Attention Residuals
• Spatula: Exploring On-Demand In-Situ Interfaces and Interaction for Attribute Control
• Focus Is All You Need: Adaptive Goal-aware Attention Orchestration for Multi-Agent Graph Systems
• Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions
• The Impact of Prompt Programming on Function-Level Code Generation
• Anchor Attention, Small Cache: Code Generation with Large Language Models
• Learning from 53.6K Real-World Developer Edits of AI-Generated Code
• AssumptionMiner: Extracting, Tracing, and Revising Implicit Assumptions in LLM Code Generation
• Do Coverage and Mutation Scores of LLM-Generated Test Suites Correlate with Their Effectiveness? (Replicability Study)
• Context as a Tool: Context Management for Long-Horizon SWE-Agents
• How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift
• PRESTO: Prefix-Aligned Tree Drafting for Diffusion Speculative Decoding
• Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation
• Broken Words, Broken Performance: Effect of Tokenization on Performance of LLMs
• Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty
• AutoBaxBuilder: Bootstrapping Code Security Benchmarking
• Artificial or Just Artful? Do LLMs Bend the Rules in Programming?
• FEM-Bench: A Structured Scientific Reasoning Benchmark for Evaluating Code-Generating LLMs
• TableGPT-R1: Advancing Tabular Reasoning Through Reinforcement Learning
• CodeSimpleQA: Scaling Factuality in Code Large Language Models
• SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios
• SoK: Understanding (New) Security Issues Across AI4Code Use Cases
• VeruSAGE: A Study of Agent-Based Verification for Rust Systems
• Propose, Solve, Verify: Self-Play Through Formal Verification
• CIFE: Code Instruction-Following Evaluation
• UCoder: Unsupervised Code Generation by Internal Probing of Large Language Models
• How Good is Post-Hoc Watermarking With Language Model Rephrasing?
• Sigma-MoE-Tiny Technical Report
• LoPA: Scaling dLLM Inference via Lookahead Parallel Decoding
• Corrective Diffusion Language Models
• Bolmo: Byteifying the Next Generation of Language Models
• Dual-Density Inference for Efficient Language Model Reasoning
• DEER: Draft with Diffusion, Verify with Autoregressive Models
• Aligning Academia with Industry: An Empirical Study of Industrial Needs and Academic Capabilities in AI-Driven Software Engineering
• An Exploratory Study of Bayesian Prompt Optimization for Test-Driven Code Generation with Large Language Models
• FrontierCS: Evolving Challenges for Evolving Intelligence
• T5Gemma 2: Seeing, Reading, and Understanding Longer
• Fast and Accurate Causal Parallel Decoding using Jacobi Forcing
• PADE: A Predictor-Free Sparse Attention Accelerator via Unified Execution and Stage Fusion
• RADAR: Accelerating Large Language Model Inference With RL-Based Dynamic Draft Trees
• Teaching Spell Checkers to Teach: Pedagogical Program Synthesis for Interactive Learning
• CAPE: Capability Achievement via Policy Execution
• ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding
• AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning
• M-GRPO: Stabilizing Self-Supervised Reinforcement Learning for Large Language Models with Momentum-Anchored Policy Optimization
• NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
• Market-Bench: Evaluating Large Language Models on Introductory Quantitative Trading and Market Dynamics
• Theoretical Foundations of GPU-Native Compilation for Rapid Code Iteration
• GLOW: Graph-Language Co-Reasoning for Agentic Workflow Performance Prediction
• Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebases
• ATLAS: Automated Toolkit for Large-Scale Verified Code Synthesis
• Linear socio-demographic representations emerge in Large Language Models from indirect cues
• Understanding Chain-of-Thought Effectiveness in Code Generation: An Empirical and Information-Theoretic Analysis
• LLaDA2.0: Scaling Up Diffusion Language Models to 100B
• Learning Unmasking Policies for Diffusion Language Models
• Multicalibration for LLM-based Code Generation
• Secure or Suspect? Investigating Package Hallucinations of Shell Command in Original and Quantized LLMs
• SimpleDevQA: Benchmarking Large Language Models on Development Knowledge QA
• PCMind-2.1-Kaiyuan-2B Technical Report
• Persian-Phi: Efficient Cross-Lingual Adaptation of Compact LLMs via Curriculum Learning
• From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs
• PrivCode: When Code Generation Meets Differential Privacy
• Yuan3.0 Ultra: A Trillion-Parameter Enterprise-Oriented MoE LLM
• Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
• Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective
• Counting Without Running: Evaluating LLMs' Reasoning About Code Complexity
• UniQL: Unified Quantization and Low-rank Compression for Adaptive Edge LLMs
• MarkTune: Improving the Quality-Detectability Trade-off in Open-Weight LLM Watermarking
• Plantain: Plan-Answer Interleaved Reasoning
• Neural steering vectors reveal dose and exposure-dependent impacts of human-AI relationships
• Rectifying LLM Thought from Lens of Optimization
• ChartAnchor: Chart Grounding with Structural-Semantic Fidelity
• Augmented Runtime Collaboration for Self-Organizing Multi-Agent Systems: A Hybrid Bi-Criteria Routing Approach
• CentaurEval: Benchmarking Human-in-the-Loop Value in Agentic Coding
• Password-Activated Shutdown Protocols for Misaligned Frontier Agents
• EduEval: A Hierarchical Cognitive Benchmark for Evaluating Large Language Models in Chinese Education
• Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match
• Generating Verifiable Chain of Thoughts from Exection-Traces
• TreeCoder: Systematic Exploration and Optimisation of Decoding and Constraints for LLM Code Generation
• BAMAS: Structuring Budget-Aware Multi-Agent Systems
• IntAttention: A Fully Integer Attention Pipeline for Efficient Edge Inference
• CafeQ: Calibration-free Quantization via Learned Transformations and Adaptive Rounding
• Can Vibe Coding Beat Graduate CS Students? An LLM vs. Human Coding Tournament on Market-driven Strategic Planning
• CDLM: Consistency Diffusion Language Models For Faster Sampling
• DRAFT-RL: Multi-Agent Chain-of-Draft Reasoning for Reinforcement Learning-Enhanced LLMs
• CLIMATEAGENT: Multi-Agent Orchestration for Complex Climate Data Science Workflows
• RPM-MCTS: Knowledge-Retrieval as Process Reward Model with Monte Carlo Tree Search for Code Generation
• DualGauge: Automated Joint Security-Functionality Benchmarking of Specification-Only Code Generation by LLMs and Coding Agents
• Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Models
• CodeR3: A GenAI-Powered Workflow Repair and Revival Ecosystem
• Xmodel-2.5: 1.3B Data-Efficient Reasoning SLM
• A²Flow: Automating Agentic Workflow Generation via Self-Adaptive Abstraction Operators
• LLM Assisted Coding with Metamorphic Specification Mutation Agent
• WavefrontDiffusion: Dynamic Decoding Schedule for Improved Reasoning
• EngChain: A Symbolic Benchmark for Verifiable Multi-Step Reasoning in Engineering
• Asking LLMs to Verify First is Almost Free Lunch
• NALA_MAINZ at BLP-2025 Task 2: A Multi-agent Approach for Bangla Instruction to Python Code Generation
• Pass@k Metric for RLVR: A Diagnostic Tool of Exploration, But Not an Objective
• InfCode: Adversarial Iterative Refinement of Tests and Patches for Reliable Software Issue Resolution
• MermaidSeqBench: An Evaluation Benchmark for LLM-to-Mermaid Sequence Diagram Generation
• GPS: General Per-Sample Prompter
• Cost-Driven Synthesis of Sound Abstract Interpreters
• Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?
• Reducing Hallucinations in LLM-Generated Code via Semantic Triangulation
• FarSkip-Collective: Unhobbling Blocking Communication in Mixture of Experts Models
• Virtual Width Networks
• Towards a Human-in-the-Loop Framework for Reliable Patch Evaluation Using an LLM-as-a-Judge
• InData: Towards Secure Multi-Step, Tool-Based Data Analysis
• ExPairT-LLM: Exact Learning for LLM Code Selection by Pairwise Queries
• Do Not Merge My Model! Safeguarding Open-Source LLMs Against Unauthorized Model Merging
• CrochetBench: Can Vision-Language Models Move from Describing to Doing in Crochet Domain?
• Think-at-Hard: Selective Latent Iterations to Improve Reasoning Language Models
• Designing LLM-based Multi-Agent Systems for Software Engineering Tasks: Quality Attributes, Design Patterns and Rationale
• MURPHY: Multi-Turn GRPO for Self Correcting Code Generation
• Ratchet: A Minimal Hygiene Recipe for Self-Evolving LLM Agents
• On the Creativity of AI Agents
• RedOne 2.0: Rethinking Domain-specific LLM Post-Training in Social Networking Services
• Retriv at BLP-2025 Task 2: Test-Driven Feedback-Guided Framework for Bangla-to-Python Code Generation
• MobileLLM-Pro Technical Report
• You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
• Route Experts by Sequence, not by Token
• Towards Resource-Efficient Multimodal Intelligence: Learned Routing among Specialized Expert Models
• OckBench: Measuring the Efficiency of LLM Reasoning
• KLASS: KL-Guided Fast Inference in Masked Diffusion Models
• Motif 2 12.7B technical report
• SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models
• Software Defined Vehicle Code Generation: A Few-Shot Prompting Approach
• From Model to Breach: Towards Actionable LLM-Generated Vulnerabilities Reporting
• Where Do LLMs Still Struggle? An In-Depth Analysis of Code Generation Benchmarks
• Benchmarking and Studying the LLM-based Agent System in End-to-End Software Development
• EDIT-Bench: Evaluating LLM Abilities to Perform Real-World Instructed Code Edits
• Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling
• VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation
• SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks
• LTD-Bench: Evaluating Large Language Models by Letting Them Draw
• Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models
• Context-Guided Decompilation: A Step Towards Re-executability
• RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Tasks
• HarnessLLM: Automatic Testing Harness Generation via Reinforcement Learning
• DPO-F+: Aligning Code Repair Feedback with Developers' Preferences
• A Comprehensive Empirical Evaluation of Agent Frameworks on Code-centric Software Engineering Tasks
• GrowthHacker: Automated Off-Policy Evaluation Optimization Using Code-Modifying LLM Agents
• Can Large Language Models Detect Real-World Android Software Compliance Violations?
• GDPR-Bench-Android: A Benchmark for Evaluating Automated GDPR Compliance Detection in Android
• Reasoning Trajectories for Socratic Debugging of Student Code: From Misconceptions to Contradictions and Updated Beliefs
• Sherlock: Reliable and Efficient Agentic Workflow Execution
• Reject Only Critical Tokens: Pivot-Aware Speculative Decoding
• ARC-GEN: A Mimetic Procedural Benchmark Generator for the Abstraction and Reasoning Corpus
• CodeAlignBench: Assessing Code Generation Models on Developer-Preferred Code Adjustments
• Agentic LLMs for REST API Test Amplification: A Comparative Study Across Cloud Applications
• Cross-Platform Evaluation of Reasoning Capabilities in Foundation Models
• CATArena: Evaluating Evolutionary Capabilities of Code Agents via Iterative Tournaments
• Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systems
• Nexus: Execution-Grounded Multi-Agent Test Oracle Synthesis
• OmniEduBench: A Comprehensive Chinese Benchmark for Evaluating Large Language Models in Education
• BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning
• Do LLMs Signal When They're Right? Evidence from Neuron Agreement
• Reasoning Curriculum: Bootstrapping Broad LLM Reasoning from Math
• Beyond Synthetic Benchmarks: Evaluating LLM Performance on Real-World Class-Level Code Generation
• Security Vulnerabilities in AI-Generated Code: A Large-Scale Analysis of Public GitHub Repositories
• QCoder Benchmark: Bridging Language Generation and Quantum Hardware through Simulator-Based Feedback
• Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents
• When is Routing Meaningful? Diversity and Robustness in Language Model Societies
• From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models
• Fewer Clarifications, Better Code: Benchmarking Cross-Session Personalized Ambiguity Adaptation in Coding Assistants
• Do Code Language Models Use Tests? A Behavioral and Representational Study of Test-Driven Code Generation
• Try Again, Don't Look Back: Blind Resampling Outperforms Self-Repair in Small Code Models
• Large Language Models for Software Engineering Diagrams: A Systematic Review of UML and ER modelling
• Cross-Model Cross-Language AI Coding Agent Performance: Accuracy and Speed of Parallel CLRS Algorithms
• EsoLang-Bench: Evaluating Genuine Reasoning in Large Language Models via Esoteric Programming Languages
• Training Language Models via Neural Cellular Automata
• RAPID: An Efficient Reinforcement Learning Algorithm for Small Language Models
• CodeSpec: Dual Executable Specifications for Agentic Long-Horizon Feature Development
• MRCoder: An Efficient Context Selecting Approach for Repository-Level Code Generation
• Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents
• REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage
• Machine-Generated, Machine-Checked Proofs for a Verified Compiler (Experience Report)
• Fine-Tuning GPT-5 for GPU Kernel Generation
• Thinking About Thinking: Evaluating Reasoning in Post-Trained Language Models
• HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing
• Ministral 3
• What Limits Agentic Systems Efficiency?
• Parallel Loop Transformer for Efficient Test-Time Computation Scaling
• CodeWiki: Evaluating AI's Ability to Generate Holistic Documentation for Large-Scale Codebases
• APTBench: Benchmarking Agentic Potential of Base LLMs During Pre-Training
• Beyond Neural Incompatibility: Easing Cross-Scale Knowledge Transfer in Large Language Models through Latent Semantic Alignment
• GIFT: Group-relative Implicit Fine Tuning Integrates GRPO with DPO and UNA
• Compiler.next: A Search-Based Compiler to Power the AI-Native Future of Software Engineering
• Variational Masked Diffusion Models
• Multi-Agent Evolve: LLM Self-Improve through Co-evolution
• The Best of N Worlds: Aligning Reinforcement Learning with Best-of-N Sampling via max@k Optimisation
• Increasing LLM Coding Capabilities through Diverse Synthetic Coding Tasks
• A Survey on LLM Mid-Training
• CodeAD: Synthesize Code of Rules for Log-based Anomaly Detection with LLMs
• Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing
• Software Engineering Agents for Embodied Controller Generation : A Study in Minigrid Environments
• Model Merging with Functional Dual Anchors
• Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like People
• Language Ranker: A Lightweight Ranking framework for LLM Decoding
• Zhyper: Factorized Hypernetworks for Conditioned LLM Fine-Tuning
• From Large to Small: Transferring CUDA Optimization Expertise via Reasoning Graph
• An Experimental Study of Real-Life LLM-Proposed Performance Improvements
• AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
• CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment
• Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs
• BlueCodeAgent: A Blue Teaming Agent Enabled by Automated Red Teaming for CodeGen AI
• AdapTrack: Constrained Decoding without Distorting LLM's Output Intent
• SpecAgent: A Speculative Retrieval and Forecasting Agent for Code Completion
• Soft-Masked Diffusion Language Models
• JT-Safe: Intrinsically Enhancing the Safety and Trustworthiness of LLMs
• TREAT: A Code LLMs Trustworthiness / Reliability Evaluation and Testing Framework
• Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Model
• Reasoning Distillation and Structural Alignment for Improved Code Generation
• QuanBench: Benchmarking Quantum Code Generation with Large Language Models
• LLMs Judge Themselves: A Game-Theoretic Framework for Human-Aligned Evaluation
• Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models
• A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
• Attention Is All You Need for KV Cache in Diffusion LLMs
• Beyond Multi-Token Prediction: Pretraining LLMs with Future Summaries
• Purifying Task Vectors in Knowledge-Aware Subspace for Model Merging
• Code-driven Number Sequence Calculation: Enhancing the inductive Reasoning Abilities of Large Language Models
• UniCode: A Framework for Generating High Quality Competitive Coding Problems
• Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models
• RLSR: Reinforcement Learning with Supervised Reward Outperforms SFT in Instruction Following
• Think Globally, Group Locally: Evaluating LLMs Using Multi-Lingual Word Grouping Games
• NOSA: Native and Offloadable Sparse Attention
• Multi-Agent Design Assistant for the Simulation of Inertial Fusion Energy
• GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models
• ConsintBench: Evaluating Language Models on Real-World Consumer Intent Understanding
• A Matter of Representation: Towards Graph-Based Abstract Code Generation
• Evaluating Arabic Large Language Models: A Survey of Benchmarks, Methods, and Gaps
• Diff-XYZ: A Benchmark for Evaluating Diff Understanding
• Do Large Language Models Respect Contracts? Evaluating and Enforcing Contract-Adherence in Code Generation
• OPLoRA: Orthogonal Projection LoRA Prevents Catastrophic Forgetting during Parameter-Efficient Fine-Tuning
• Representation-Based Exploration for Language Models: From Test-Time to Post-Training
• Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Models
• Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation
• Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief States
• DND: Boosting Large Language Models with Dynamic Nested Depth
• AMiD: Knowledge Distillation for LLMs with α-mixture Assistant Distribution
• Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization
• Testing and Enhancing Multi-Agent Systems for Robust Code Generation
• Rethinking LLM Evaluation: Can We Evaluate LLMs with 200x Less Data?
• ZeroGR: A Generalizable and Scalable Framework for Zero-Shot Generative Retrieval
• Learning to Guarantee Type Correctness in Code Generation through Type-Guided Program Synthesis
• Debiasing LLMs by Masking Unfairness-Driving Attention Heads
• Failure-Driven Workflow Refinement
• StelLA: Subspace Learning in Low-rank Adaptation using Stiefel Manifold
• On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
• Understanding and Exploiting Weight Update Sparsity for Communication-Efficient Distributed RL
• LiveOIBench: Can Large Language Models Outperform Human Contestants in Informatics Olympiads?
• McMining: Automated Discovery of Misconceptions in Student Code
• InteractScience: Programmatic and Visually-Grounded Evaluation of Interactive Scientific Demonstration Code Generation
• A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System
• ICL-Router: In-Context Learned Model Representations for LLM Routing
• Constraints-of-Thought: A Framework for Constrained Reasoning in Language-Model-Guided Search
• CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards
• Which Heads Matter for Reasoning? RL-Guided KV Cache Compression
• dInfer: An Efficient Inference Framework for Diffusion Language Models
• Guided Star-Shaped Masked Diffusion
• Robust Heuristic Algorithm Design with LLMs
• Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimization
• AppForge: From Assistant to Independent Developer -- Are GPTs Ready for Software Development?
• MOSAIC: Multi-agent Orchestration for Task-Intelligent Scientific Coding
• AgentAsk: Multi-Agent Systems Need to Ask
• Fortifying LLM-Based Code Generation with Graph-Based Reasoning on Secure Coding Practices
• More Data or Better Data? A Critical Analysis of Data Selection and Synthesis for Mathematical Reasoning
• Accelerating Diffusion LLM Inference via Local Determinism Propagation
• Mid-Training of Large Language Models: A Survey
• Do LLMs Know They Are Being Tested? Evaluation Awareness and Incentive-Sensitive Failures in GPT-OSS-20B
• GRACE: A Language Model Framework for Explainable Inverse Reinforcement Learning
• What MLLMs Learn about When they Learn about Multimodal Reasoning
• Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs
• Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levels
• Off-Trajectory Reasoning: Can LLMs Collaborate on Reasoning Trajectory?
• RECODE-H: A Benchmark for Research Code Development with Interactive Human Feedback
• SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
• VeriEquivBench: An Equivalence Score for Ground-Truth-Free Evaluation of Formally Verifiable Code
• Fine-Tuning Masked Diffusion for Provable Self-Correction
• Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs
• Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models
• SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs
• FreshBrew: A Benchmark for Evaluating AI Agents on Java Code Migration
• FedSRD: Sparsify-Reconstruct-Decompose for Communication-Efficient Federated Large Language Models Fine-Tuning
• Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches
• LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning
• Inoculation Prompting: Instructing LLMs to misbehave at train-time improves test-time alignment
• Machine Learning for Detection and Analysis of Novel LLM Jailbreaks
• Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time
• SPOGW: a Score-based Preference Optimization method via Group-Wise comparison for workflows
• When Names Disappear: Revealing What LLMs Actually Understand About Code
• GramTrans: A Better Code Representation Approach in Code Generation
• AutoMaAS: Self-Evolving Multi-Agent Architecture Search for Large Language Models
• Model-Agnostic Correctness Assessment for LLM-Generated Code via Dynamic Internal Representation Selection
• Mechanistic Interpretability of Code Correctness in LLMs via Sparse Autoencoders
• Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks
• Can Emulating Semantic Translation Help LLMs with Code Translation? A Study Based on Pseudocode
• AP2O-Coder: Adaptively Progressive Preference Optimization for Reducing Compilation and Runtime Errors in LLM-Generated Code
• Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerability
• CodeChemist: Functional Knowledge Transfer for Low-Resource Code Generation via Test-Time Scaling
• Programming Puzzles
• Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling
• Towards Ecologically Valid LLM Benchmarks: Understanding and Designing Domain-Centered Evaluations for Journalism Practitioners
• Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models
• DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively
• Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmark
• Revealing the Power of Post-Training for Small Language Models via Knowledge Distillation
• dParallel: Learnable Parallel Decoding for dLLMs
• AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Size
• Communication-Efficient and Accurate Approach for Aggregation in Federated Low-Rank Adaptation
• Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in Its Latent Thoughts
• Interactive Learning for LLM Reasoning
• Lita: Light Agent Uncovers the Agentic Coding Capabilities of LLMs
• Expert Merging: Model Merging with Unsupervised Expert Alignment and Importance-Guided Layer Chunking
• Learning to Reason as Action Abstractions with Scalable Mid-Training RL
• RFG: Test-Time Scaling for Diffusion Large Language Model Reasoning with Reward-Free Guidance
• MixtureVitae: Open Web-Scale Pretraining Dataset With High Quality Instruction and Reasoning Data Built from Permissive-First Text Sources
• Adaptive Test-Time Reasoning via Reward-Guided Dual-Phase Search
• UniAPL: A Unified Adversarial Preference Learning Framework for Instruct-Following
• Towards Reliable Generation of Executable Workflows by Foundation Models
• Bridging Developer Instructions and Code Completion Through Instruction-Aware Fill-in-the-Middle Paradigm
• SemGuard: Real-Time Semantic Evaluator for Correcting LLM-Generated Code
• Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agents
• Speculative Verification: Exploiting Information Gain to Refine Speculative Decoding
• MAS²: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems
• ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Models
• TENET: Leveraging Tests Beyond Validation for Code Generation
• Watermarking Diffusion Language Models
• InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptation
• Short window attention enables long-term memorization
• Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding
• Sequential Diffusion Language Models
• ByteSized32Refactored: Towards an Extensible Interactive Text Games Corpus for LLM World Modeling and Evaluation
• HiPO: Hybrid Policy Optimization for Dynamic Reasoning in LLMs
• Beyond Benchmarks: Understanding Mixture-of-Experts Models through Internal Mechanisms
• Anchored Supervised Fine-Tuning
• Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models
• Fast Thinking for Large Language Models
• SolContractEval: A Benchmark for Evaluating Contract-Level Solidity Code Generation
• Mapping Overlaps in Benchmarks through Perplexity in the Wild
• Planner Aware Path Learning in Diffusion Language Models Training
• A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Models
• CATMark: A Context-Aware Thresholding Framework for Robust Cross-Task Watermarking in Large Language Models
• Local Success Does Not Compose: Benchmarking Large Language Models for Compositional Formal Verification
• CoDA: Coding LM via Diffusion Adaptation
• Protocode: Prototype-Driven Interpretability for Code Generation in LLMs
• Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models
• d²Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
• Critique-Coder: Enhancing Coder Models by Critique Reinforcement Learning
• InfiAgent: Self-Evolving Pyramid Agent Framework for Infinite Scenarios
• Meta-Awareness Enhances Reasoning Models: Self-Alignment Reinforcement Learning
• Stochastic activations
• FeatBench: Evaluating Coding Agents on Feature Implementation for Vibe Coding
• SK2Decompile: LLM-based Two-Phase Binary Decompilation from Skeleton to Skin
• Reinforcement Learning-Guided Chain-of-Draft for Token-Efficient Code Generation
• AgentPack: A Dataset of Code Changes, Co-Authored by Agents and Humans
• RobustFlow: Towards Robust Agentic Workflow Generation
• VibeCodeHPC: An Agent-Based Iterative Prompting Auto-Tuner for HPC Code Generation Using LLMs
• Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Data
• InvBench: Can LLMs Accelerate Program Verification with Invariant Synthesis?
• StyleBench: Evaluating thinking styles in Large Language Models
• Mixture of Thoughts: Learning to Aggregate What Experts Think, Not Just What They Say
• RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?
• d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation
• When Instructions Multiply: Measuring and Estimating LLM Capabilities of Multiple Instructions Following
• LayoutAgent: A Vision-Language Agent Guided Compositional Diffusion for Spatial Layout Planning
• Beyond Language Barriers: Multi-Agent Coordination for Multi-Language Code Generation
• Intuition to Evidence: Measuring AI's True Impact on Developer Productivity
• An Empirical Study of Coordination Mode as the First-Class Citizen in From-Scratch Multi-Agent Coding
• Σ-Mem: An Online Reliability Memory for LLM-based Multi-Agent Systems
• When Specifications Conflict: A Symmetry-Based Framework for Measuring LLM Preferences
• Specification-Guided Synthesis of Deadlock-Free Communication Protocol Refinements with Large Language Models
• Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents
• Beyond Rephrasing: Book-Level Organization Improves Synthetic Textbook Data for Mid-Training
• SWE-NFI: Studying and Benchmarking Coding Agents for Non-Functional Improvements
• HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMs
• KernelGenBench: A Multi-Source and Multi-Chip Benchmark for LLM-based Kernel Generation
• Multi-Head Attention Residuals
• DFlash: Block Diffusion for Flash Speculative Decoding
• On the Soundness and Consistency of LLM Agents for Executing Test Cases Written in Natural Language
• When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models
• SR-Eval: Evaluating LLMs on Code Generation under Stepwise Requirement Refinement
• Benchmark Profiling: Mechanistic Diagnosis of LLM Benchmarks
• Program Synthesis via Test-Time Transduction
• MapCoder-Lite: Distilling Multi-Agent Coding into a Single Small LLM
• Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphs
• PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
• SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
• Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation
• Generating High-Quality Datasets for Code Editing via Open-Source Language Models
• CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics
• Analyzing and Mitigating Surface Bias in Code Evaluation Metrics
• School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMs
• CARGO: A Framework for Confidence-Aware Routing of Large Language Models
• Reveal and Release: Iterative LLM Unlearning with Self-generated Data
• Position: Thematic Analysis of Unstructured Clinical Transcripts with Large Language Models
• (P)rior(D)yna(F)low: A Priori Dynamic Workflow Construction via Multi-Agent Collaboration
• Masked Diffusion Models as Energy Minimization
• Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearning
• ZERA: Zero-init Instruction Evolving Refinement Agent -- From Zero Instructions to Structured Prompts via Principle-based Optimization
• A Study on Thinking Patterns of Large Reasoning Models in Code Generation
• CLMTracing: Black-box User-level Watermarking for Code Language Model Tracing
• Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning
• Evaluating Classical Software Process Models as Coordination Mechanisms for LLM-Based Software Generation
• Explain Before You Answer: A Survey on Compositional Visual Reasoning
• Automating Code Generation for Semiconductor Equipment Control from Developer Utterances with LLMs
• SCoGen: Scenario-Centric Graph-Based Synthesis of Real-World Code Problems
• From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
• UniPar: A Unified LLM-Based Framework for Parallel and Accelerated Code Translation in HPC
• Rethinking Technology Stack Selection with AI Coding Proficiency
• Difficulty-Aware Agentic Orchestration for Query-Specific Multi-Agent Workflows
• Beyond Autoregression: An Empirical Study of Diffusion Large Language Models for Code Generation
• RefactorCoderQA: Benchmarking LLMs for Multi-Domain Coding Question Solutions in Cloud and Edge Deployment
• SciML Agents: Write the Solver, Not the Solution
• MCBP: A Memory-Compute Efficient LLM Inference Accelerator Leveraging Bit-Slice-enabled Sparsity and Repetitiveness
• SWE-Mirror: Scaling Issue-Resolving Datasets by Mirroring Issues Across Repositories
• ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation
• SCoder: Iterative Self-Distillation for Bootstrapping Small-Scale Data Synthesizers to Empower Code LLMs
• Language Self-Play For Data-Free Training
• Astra: A Multi-Agent System for GPU Kernel Performance Optimization
• Learning to Synthesize Programs as Interpretable and Generalizable\n Policies
• LAMDAS: LLM as an Implicit Classifier for Domain-specific Data Selection
• Ban&Pick: Ehancing Performance and Efficiency of MoE-LLMs via Smarter Routing
• Stack Overflow Is Not Dead Yet: Crowd Answers Still Matter
• GRACE: Graph-Guided Repository-Aware Code Completion through Hierarchical Code Fusion
• Using Contrastive Learning to Improve Two-Way Reasoning in Large Language Models: The Obfuscation Task as a Case Study
• Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts
• Symbolic Graphics Programming with Large Language Models
• Set Block Decoding is a Language Model Inference Accelerator
• On Robustness and Reliability of Benchmark-Based Evaluation of LLMs
• app.build: A Production Framework for Scaling Agentic Prompt-to-App Generation with Environment Scaffolding
• AetherCode: Evaluating LLMs' Ability to Win In Premier Programming Competitions
• Mixture-of-Clustered-Experts: Advancing Expert Specialization and Generalization in Instruction Tuning
• Efficient Training-Free Online Routing for High-Volume Multi-LLM Serving
• LLM Agents for Generating Microservice-based Applications: how complex is your specification?
• Implicit Reasoning in Large Language Models: A Comprehensive Survey
• GEM: A Scale-Aware and Distribution-Sensitive Sparse Fine-Tuning Framework for Effective Downstream Adaptation
• Inducing Faithfulness in Structured Reasoning via Counterfactual Sensitivity
• Aligning Requirement for Large Language Model's Code Generation
• Dream-Coder 7B: An Open Diffusion Language Model for Code
• Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling
• From CVE Entries to Verifiable Exploits: An Automated Multi-Agent Framework for Reproducing CVEs
• TinyMusician: On-Device Music Generation with Knowledge Distillation and Mixed Precision Quantization
• Analysis of Error Sources in LLM-based Hypothesis Search for Few-Shot Rule Induction
• Can Multi-turn Self-refined Single Agent LMs with Retrieval Solve Hard Coding Problems?
• ChopChop: a Programmable Framework for Semantically Constraining the Output of Language Models
• Balanced Actor Initialization: Stable RLHF Training of Distillation-Based Reasoning Models
• HiVA: Self-organized Hierarchical Variable Agent via Goal-driven Semantic-Topological Evolution
• Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning
• Active Learning for Neurosymbolic Program Synthesis
• 11Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysis
• Diffusion Language Models Know the Answer Before Decoding
• PSO-Merging: Merging Models Based on Particle Swarm Optimization
• Benchmarking Hindi LLMs: A New Suite of Datasets and a Comparative Analysis
• Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
• ReST-RL: Achieving Accurate Code Reasoning of LLMs with Optimized Self-Training and Decoding
• Alignment with Fill-In-the-Middle for Enhancing Code Generation
• Prebiotic Functional Programs: Endogenous Selection in an Artificial Chemistry
• Guidelines for Empirical Studies in Software Engineering involving Large Language Models
• LongReasonArena: A Long Reasoning Benchmark for Large Language Models
• Predicting the Order of Upcoming Tokens Improves Language Modeling
• SynthCoder: A Synthetical Strategy to Tune LLMs for Code Completion
• GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging
• UltraMemV2: Memory Networks Scaling to 120B Parameters with Superior Long-Context Learning
• Optimal Sparsity of Mixture-of-Experts Language Models for Reasoning Tasks
• Beyond Benchmark: LLMs Evaluation with an Anthropomorphic and Value-oriented Roadmap
• Latent Self-Consistency for Reliable Majority-Set Selection in Short- and Long-Answer Reasoning
• A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code
• Detecting and Characterizing Planning in Language Models
• Dream 7B: Diffusion Large Language Models
• Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset
• Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs
• NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model
• COMPASS: A Multi-Dimensional Benchmark for Evaluating Code Generation in Large Language Models
• DPad: Efficient Diffusion Language Models with Suffix Dropout
• PC-Sampler: Position-Aware Calibration of Decoding Bias in Masked Diffusion Models
• Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation
• Optimal Neural Program Synthesis from Multimodal Specifications
• Is GPT-OSS Good? A Comprehensive Evaluation of OpenAI's Latest Open Source Models
• Uncovering Systematic Failures of LLMs in Verifying Code Against Natural Language Specifications
• Optimizing Token Choice for Code Watermarking: An RL Approach
• AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities
• Thinking Inside the Mask: In-Place Prompting in Diffusion LLMs
• Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
• Apriel-Nemotron-15B-Thinker
• SaraCoder: Orchestrating Semantic and Structural Cues for Resource-Optimized Repository-Level Code Completion
• Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments
• SAEMark: Multi-bit LLM Watermarking with Inference-Time Scaling
• AdaptFlow: Adaptive Workflow Optimization via Meta-Learning
• \(X\)-evolve: Solution space evolution powered by large language models
• Dynamic Benchmark Construction for Evaluating Large Language Models on Real-World Codes
• SABER: Switchable and Balanced Training for Efficient LLM Reasoning
• Devstral: Fine-tuning Language Models for Coding Agent Applications
• Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
• DP-LLM: Runtime Model Adaptation with Dynamic Layer-wise Precision Assignment
• RTTC: Reward-Guided Collaborative Test-Time Compute
• A Novel Architecture for Symbolic Reasoning with Decision Trees and LLM Agents
• MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
• CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RL
• Posterior-GRPO: Rewarding Reasoning Processes in Code Generation
• Klear-CodeTest: Scalable Test Case Generation for Code Reinforcement Learning
• Agnostics: Learning to Code in Any Programming Language via Reinforcement with a Universal Learning Environment
• CARD: A Cache-Assisted Parallel Speculative Decoding Framework via Query-and-Correct Paradigm for Accelerating LLM Inference
• StackPilot: Autonomous Function Agents for Scalable and Environment-Free Code Execution
• Tensorized Clustered LoRA Merging for Multi-Task Interference
• More Than a Score: Probing the Impact of Prompt Specificity on LLM Code Generation
• Refining Critical Thinking in LLM Code Generation: A Faulty Premise-based Evaluation Framework
• LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinking
• A System Model Generation Benchmark from Natural Language Requirements
• RegMean++: Enhancing Effectiveness and Generalization of Regression Mean for Model Merging
• Tool-integrated Reinforcement Learning for Repo Deep Search
• MRG-Bench: Evaluating and Exploring the Requirements of Context for Repository-Level Code Generation
• CTTS: Collective Test-Time Scaling
• Polymath: A Self-Optimizing Agent with Dynamic Hierarchical Workflow
• Meta-RAG on Large Codebases Using Code Summarization
• MicroMix: Efficient Mixed-Precision Quantization with Microscaling Formats for Large Language Models
• EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
• What's in a Proof? Analyzing Expert Proof-Writing Processes in F* and Verus
• Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models
• EMA Without the Lag: Bias-Corrected Iterate Averaging Schemes
• CFDagent: A Language-Guided, Zero-Shot Multi-Agent System for Complex Flow Simulation
• Geak: Introducing Triton Kernel AI Agent & Evaluation Benchmarks
• AutoBridge: Automating Smart Device Integration with Centralized Platform
• DSBC : Data Science task Benchmarking with Context engineering
• Doctor Sun: A Bilingual Multimodal Large Language Model for Biomedical AI
• IFEvalCode: Controlled Code Generation
• UnsafeChain: Enhancing Reasoning Model Safety via Hard Cases
• PhaseNAS: Language-Model Driven Architecture Search with Dynamic Phase Adaptation
• Kimi K2: Open Agentic Intelligence
• When Prompts Go Wrong: Evaluating Code Model Robustness to Ambiguous, Contradictory, and Incomplete Task Descriptions
• Language model benchmark [wikipedia]

Discussions

• Program Synthesis with Large Language Models [hn, 63 points, 20 comments]

Related

• CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
• Jigsaw: Large Language Models meet Program Synthesis
• L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models
• Function-constrained Program Synthesis
• Low-Cost Language Models: Survey and Performance Evaluation on Python Code Generation