Evaluating Large Language Models Trained on Code

2021/07/07 by Mark Chen, Chen, Mark, Jerry Tworek +122 · 9 voices · 1215 citations
Computer Science · #Parallel Computing and Optimization Techniques #Software Engineering Research #Topic Modeling #cs.LG

paper · pdf · doi:10.48550/arxiv.2107.03374

openalex publication_date 2021/07/07 · openalex created_date 2021/07/19 · openalex updated_date 2026/07/30

Abstract

I created a basic proof of concept of LMVM (Language Model Virtual Machine), a command line toolchain that English instructions into native binaries without intermediary high-level language compilation. The command line uses a large language model (LLM) as IR generation backend, producing LLVM Intermediate Representation (IR) directly from user instructions coding. The IR is compiled and linked to machine code (-o files) via `clang` (with an `llc`/`lld` fallback on POSIX hosts) and executed on bare metal, within Docker sandboxes, on AWS Lambda, or as standalone HTTP servers. How It Works (5 Steps) 1. You type an instruction - e.g. "Create a rest api running on localhost port 3000 and return hello world" 2. LMVM checks your computer - It figures out if you're on Windows/Mac/Linux, what CPU you have (Intel vs ARM), and makes sure clang (the compiler) is installed. If not, it offers to install it for you (with your permission). 3. An AI writes LLVM IR - Instead of writing C or Python, the LLM writes LLVM IR -- a low-level assembly-like language that sits between source code and machine code. The prompt includes your OS details so the AI generates code that works on your platform (e.g. Winsock on Windows vs POSIX sockets on Mac/Linux). 4. Clang turns IR into a native binary - clang compiles the IR and links it with system libraries (PostgreSQL, OpenSSL, SQLite, etc.) into a real executable. One step, no extra tools needed. 5. It runs - The binary executes directly on your CPU. If it's a Docker target, it runs sandboxed. If it's Lambda, it packages for AWS. 2. Why we care : Compiled native binaries outperform interpreted languages by 10-50x in CPU usage and 5-15x in memory for compute-heavy tasks, because interpreters add overhead with every instruction they process and their single-threaded execution model blocks true parallelism. Few runtimes that compile code on the fly, the gap narrows to 2-5x in CPU since those systems optimize hot code paths over time -- but native code still starts faster, uses less memory, and can run true parallel threads instead of juggling everything through a single event loop. The biggest gains show up in recursive and number-crunching workloads (where the cost of "figuring out what to do next" dominates) and memory-heavy tasks (where automatic garbage collection and wrapping every value in an object bloats memory usage). I/O-bound tasks show the smallest differences, because the operating system does the real work regardless of what language runtime sits on top. This also means we need to think carefully about maintainability, especially now that AI agents can generate code as a complete black box sometimes even producing low-level output that humans rarely inspect. 3. System Architecture [CLI] → [Toolchain] → [LLM Client] → [Compiler] → [Runner] → Output ↑ | Error Feedback | Self‑Correction Loop (x10) 3.1 CLI and Interactive Setup The command-line interface (built on Commander.js) exposes four commands: `run` (compile and execute), `compile` (IR generation only), `build` (compile existing IR), and `targets` (list output targets). 3.2 LLM Client The LLM client wraps an OpenAI-compatible API, supporting providers examples (OpenAI, Groq, Together AI, OpenRouter, and local llm instances). The command line agent constructs prompts from two sources: a system prompt (loaded from `Prompt.md`, ~4000 tokens) containing LLVM IR generation rules, security constraints, and code templates; and a user prompt containing a platform descriptor (host OS, exact target triple, sockets dialect), the instruction, target, and error context for retry attempts. A secondary method, is used by the toolchain bootstrapper to obtain package-manager commands for the detected host. 3.3 Platform Abstraction and Toolchain Bootstrapper Since LMVM builds real native programs, it has to know exactly what computer it's running on -- the same code won't work the same way on Windows vs Mac vs Linux. So a platform module figures out three things: 1. The "target triple" -- a label that tells the compiler exactly which OS and CPU to build for (like x86₆4-pc-windows-msvc for 64-bit Windows, or arm64-apple-darwin for an M-series Mac). 2. How to do networking -- Windows uses its own socket system (Winsock), while Mac and Linux use the standard POSIX approach. The code has to match. 3. What package manager is available -- winget/choco/scoop on Windows, brew on Mac, apt/dnf/pacman on Linux. This matters if LMVM needs to install something. All of this gets described in plain text and injected into the AI's prompt, so it writes code that actually compiles on your machine instead of assuming you're on Linux. Before talking to the AI, LMVM runs a checklist to make sure your computer can actually compile things: 1. PATH fix - If you just installed clang but your terminal hasn't refreshed yet, LMVM detects the install location and patches its own PATH so it finds clang right away, no need to restart your terminal. 2. Is clang there? - It checks that the compiler is actually available. 3. Can clang actually build a program? (Windows only) -- On Windows, having clang isn't enough -- it also needs the MSVC linker and Windows SDK behind it. LMVM compiles a tiny test program to verify the full chain works. If it fails, the Visual Studio Build Tools are probably missing. 4. Install with user's permission - If anything is missing, LMVM asks the AI for the right install commands, shows them to you, and waits for your yes before running anything. Then it checks again. If the toolchain still can't work after all that, LMVM stops early and tells you exactly what to do (install manually, open a new terminal, or use Docker instead). It doesn't waste your AI API calls retrying something a code fix can't solve. 3.4 Code Compiler Compilation is driven by `clang`, which lowers the `.ll` and invokes the platform linker in a single step: Primary path (all platforms) - `clang output.ll <lib-flags> -o <binary>`. Library flags are platform-aware: POSIX hosts use `-lpq`, `-lcurl`, `-lssl`, etc.; Windows links `ws2<sub>3</sub>2` for socket/server targets and relies on the MSVC C runtime. For the AWS Lambda target, static linking is applied. POSIX fallback - If `clang` is unavailable on macOS/Linux, the system falls back to the classic cascade: the LLVM Static Compiler (`llc -filetype=obj -relocation-model=pic`) to produce a position-independent object file, then `ld.lld` (`-flavor gnu`), the system C compiler (`cc`), or `clang` as a linker. A 60-second timeout bounds each invocation. The compiler maps library names to their respective linker flags through platform-specific lookup tables, covering PostgreSQL (libpq), HTTP (libcurl), JSON (cJSON), SSL/TLS (OpenSSL), image processing (libpng, libjpeg), databases (libsqlite3, libmysqlclient), compression (zlib), encryption (libcrypto), caching (hiredis), and message queues (librdkafka). 3.5 Execution Runner The runner dispatches binary execution based on the detected target like native , docker or AWS or GCR. 3.6 Self-Correction Loop When compilation or execution fails, the runtime updates a VM state object (`VMState`) with the error type (compiler/linker/runtime), error message, error line number, previous IR, and a compressed memory object (`VMMemory`) tracking accumulated errors and resolved dependencies. This enriched context is fed back to the LLM in the next attempt, enabling targeted correction. The loop terminates after a configurable maximum number of attempts (default: 10). 3.7 Security Layer LMVM employs defense-in-depth across three layers: 1. AI rules -- The instructions sent to the AI explicitly say: don't generate code that runs shell commands with user input, don't write files outside the allowed folder, don't read sensitive system files, don't hardcode passwords, and don't allocate unlimited memory. Think of this as a "don't do bad things" rulebook for the AI. 2. Sandbox -- When running in Docker, the program is locked down. A seccomp profile blocks the most dangerous system calls -- things like creating new processes, mounting drives, rebooting the machine, loading kernel modules, or snooping on other programs. Docker also limits the program to 512 MB of RAM and 1 CPU core, so it can't eat all your resources. This follows NIST security guidelines. 3. Input protection -- When LMVM runs the compiler or your program, it passes arguments as a structured list, not a single string. This means a hacker can't sneak in shell commands the way they might with string concatenation (the classic "shell injection" attack). The one exception is the toolchain installer, which does run shell commands -- but it always shows you the exact commands first and waits for your approval before running them. API keys are hidden in console output, and data buffers are capped at 10 MB to prevent memory explosions Appendix : As a demo using command LMVMcli command line we created a basic rest api directly as a machice code. Generated IR by GPT

Citations

• Learning to Execute
• Semi-supervised Sequence Learning
• Neural GPUs Learn Algorithms
• Pixel Recurrent Neural Networks
• Latent Predictor Networks for Code Generation
• DeepCoder: Learning to Write Programs
• A parallel corpus of Python functions and documentation strings for automated code documentation and code generation
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• Scaling Laws for Neural Language Models
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
• Unsupervised Translation of Programming Languages
• Learning to summarize from human feedback
• CodeBLEU: a Method for Automatic Evaluation of Code Synthesis
• CodeBERT: A Pre-Trained Model for Programming and Natural Languages
• The Pile: An 800GB Dataset of Diverse Text for Language Modeling
• CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding\n and Generation
• Zero-Shot Text-to-Image Generation
• On the Dangers of Stochastic Parrots
• Learning Transferable Visual Models From Natural Language Supervision
• Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
• MERLOT: Multimodal Neural Script Knowledge Models

Cited by

• PurifyGen: A Risk-Discrimination and Semantic-Purification Model for Safe Text-to-Image Generation
• AKG kernel Agent: A Multi-Agent Framework for Cross-Platform Kernel Synthesis
• Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Data
• A Probabilistic Framework for LLM-Based Model Discovery
• Post-Training Quantization of OpenPangu Models for Efficient Deployment on Atlas A2
• Anka: A Domain-Specific Language for Reliable LLM Code Generation
• Scoring, Reasoning, and Selecting the Best! Ensembling Large Language Models via a Peer-Review Process
• Is Chain-of-Thought Really Not Explainability? Chain-of-Thought Can Be Faithful without Hint Verbalization
• ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions
• Diversity or Precision? A Deep Dive into Next Token Prediction
• BenCSSmark: Making the Social Sciences Count in LLM Research
• FasterPy: An LLM-based Code Execution Efficiency Optimization Framework
• WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference
• Rethinking the Capability of Fine-Tuned Language Models for Automated Vulnerability Repair
• M2G-Eval: Enhancing and Evaluating Multi-granularity Multilingual Code Generation
• AFA-LoRA: Enabling Non-Linear Adaptations in LoRA with Activation Function Annealing
• Efficient Multi-Model Orchestration for Self-Hosted Large Language Models
• Beyond Single Bugs: Benchmarking Large Language Models for Multi-Vulnerability Detection
• GPU Kernel Optimization Beyond Full Builds: An LLM Framework with Minimal Executable Programs
• HELIOS: An LLM-Driven Autonomous Indirect Trajectory Optimization Agent
• DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention
• Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion
• Attention Residuals
• Same Prompt, Different Outcomes: Evaluating the Reproducibility of Data Analysis by LLMs
• ATLAS: Automated Approximation of Transformers for Efficient Homomorphic Inference in One Hour
• Spatula: Exploring On-Demand In-Situ Interfaces and Interaction for Attribute Control
• CircuitWeave: Topology-Behavior Alignment for Executable Multimodal RTL Generation
• The Illusion of Secure LLM Code: Closing the Security Gap via Iterative Reprompting
• Poster: Rethinking Security in LLM Code Generation through Real-World Risk Scenarios
• Verbalized Particle Posterior: Bayesian Inference over Natural Language Hypotheses
• Generative Artificial Intelligence (GenAI) to convert images of queuing networks into verifiable simulation models: an open-weight LLM workflow approach
• DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoning
• Early-Stage Prediction of Review Effort in AI-Generated Pull Requests
• Bridging Compute- and Data-Optimal Pretraining
• Psychological Competence as a Missing Dimension in AI Evaluation
• A Control System, a Dataset, and a Recipe for Making Frozen LLM Agents Learn a Domain
• Specula: Scaling formal specifications for autonomous model checking of system code
• Specification-Driven DevOps for Multi-Service Environments
• Toward an Organizational Science of Multi-Agent LLM Systems: Decoupling Who, How, and Which Algorithm
• SecDrift: Measuring Sector-Conditioned Security Drift in AI-Generated Code
• Learning from 53.6K Real-World Developer Edits of AI-Generated Code
• Beyond "What to Retrieve": Uncertainty in Retrieval-Augmented Code Generation
• VClare: Resolving Imperfect Specifications in LLM-Based Verilog Generation
• AssumptionMiner: Extracting, Tracing, and Revising Implicit Assumptions in LLM Code Generation
• PRAXIS: Integrating Program Analysis with Observability for Root-Cause Analysis
• Do Coverage and Mutation Scores of LLM-Generated Test Suites Correlate with Their Effectiveness? (Replicability Study)
• Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit Tests
• AdaKP: Online Adaptive Knowledge-Point Selection for Reasoning-Oriented Reinforcement Learning
• Context as a Tool: Context Management for Long-Horizon SWE-Agents
• How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift
• PRESTO: Prefix-Aligned Tree Drafting for Diffusion Speculative Decoding
• Evaluating LLMs as Interpretable Controllers for Dynamical Systems
• Precise Debugging Benchmark: Is Your Model Debugging or Regenerating?
• PatchGuru: Patch Oracle Inference from Natural Language Artifacts
• CricBench: A Multilingual Benchmark for Evaluating LLMs in Cricket Analytics
• Analyzing Code Injection Attacks on LLM-based Multi-Agent Systems in Software Development
• How Do Agents Perform Code Optimization? An Empirical Study
• What Makes a GitHub Issue Ready for Copilot?
• Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty
• Your Reasoning Benchmark May Not Test Reasoning: Revealing Perception Bottleneck in Abstract Reasoning Benchmarks
• Measuring all the noises of LLM Evals
• AutoBaxBuilder: Bootstrapping Code Security Benchmarking
• Artificial or Just Artful? Do LLMs Bend the Rules in Programming?
• Assessing the Software Security Comprehension of Large Language Models
• NotSoTiny: A Large, Living Benchmark for RTL Code Generation
• FEM-Bench: A Structured Scientific Reasoning Benchmark for Evaluating Code-Generating LLMs
• Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs
• Generative Digital Twins: Vision-Language Simulation Models for Executable Industrial Systems
• Toward Explaining Large Language Models in Software Engineering Tasks
• TableGPT-R1: Advancing Tabular Reasoning Through Reinforcement Learning
• Synthesizing Procedural Memory: Challenges and Architectures in Automated Workflow Generation
• Neuron-Guided Interpretation of Code LLMs: Where, Why, and How?
• AXIOM: Benchmarking LLM-as-a-Judge for Code via Rule-Based Perturbation and Multisource Quality Calibration
• The Seismic Wavefield Common Task Framework
• Small Language Models as Compiler Experts: Auto-Parallelization for Heterogeneous Systems
• Population-Evolve: a Parallel Sampling and Evolutionary Method for LLM Math Reasoning
• Generation of Programmatic Rules for Document Forgery Detection Using Large Language Models
• Vibe Reasoning: Eliciting Frontier AI Mathematical Capabilities -- A Case Study on IMO 2025 Problem 6
• AI Code in the Wild: Measuring Security Risks and Ecosystem Shifts of AI-Generated Code in Modern Software
• CosmoCore-Evo: Evolutionary Dream-Replay Reinforcement Learning for Adaptive Code Generation
• SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios
• SoK: Understanding (New) Security Issues Across AI4Code Use Cases
• VeruSAGE: A Study of Agent-Based Verification for Rust Systems
• Intelligent Human-Machine Partnership for Manufacturing: Enhancing Warehouse Planning through Simulation-Driven Knowledge Graphs and LLM Collaboration
• Software Vulnerability Management in the Era of Artificial Intelligence: An Industry Perspective
• External Hippocampus: Topological Cognitive Maps for Guiding Large Language Model Reasoning
• Propose, Solve, Verify: Self-Play Through Formal Verification
• LLM Agents Implement an NLG System from Scratch: Building Interpretable Rule-Based RDF-to-Text Generators
• Holistic Evaluation of State-of-the-Art LLMs for Code Generation
• From Prompt to Product: A Human-Centered Benchmark of Agentic App Generation Systems
• ReX-MLE: The Autonomous Agent Benchmark for Medical Imaging Challenges
• CIFE: Code Instruction-Following Evaluation
• SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositories
• UCoder: Unsupervised Code Generation by Internal Probing of Large Language Models
• Governance-Aware Hybrid Fine-Tuning for Multilingual Large Language Models
• Bridging Natural Language and Formal Specification--Automated Translation of Software Requirements to LTL via Hierarchical Semantics Decomposition Using LLMs
• Reasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMs
• Trust-Region Adaptive Policy Optimization
• LLM-HPC++: Evaluating LLM-Generated Modern C++ and MPI+OpenMP Codes for Scalable Mandelbrot Set Computation
• How Good is Post-Hoc Watermarking With Language Model Rephrasing?
• DataFlow: An LLM-Driven Framework for Unified Data Preparation and Workflow Automation in the Era of Data-Centric AI
• Refusal Steering: Fine-grained Control over LLM Refusal Behaviour for Sensitive Topics
• A Systematic Study of Code Obfuscation Against LLM-based Vulnerability Detection
• Sigma-MoE-Tiny Technical Report
• DualGuard: Dual-stream Large Language Model Watermarking Defense against Paraphrase and Spoofing Attack
• A Neurosymbolic Approach to Loop Invariant Generation via Weakest Precondition Reasoning
• DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Models
• Corrective Diffusion Language Models
• Bolmo: Byteifying the Next Generation of Language Models
• Revisiting Task-Oriented Dataset Search in the Era of Large Language Models: Challenges, Benchmark, and Solution
• Aligning Academia with Industry: An Empirical Study of Industrial Needs and Academic Capabilities in AI-Driven Software Engineering
• Beyond Majority Voting: Towards Fine-grained and More Reliable Reward Signal for Test-Time Reinforcement Learning
• Beyond Fast and Slow: Cognitive-Inspired Elastic Reasoning for Large Language Models
• Quantifying Return on Security Controls in LLM Systems
• FrontierCS: Evolving Challenges for Evolving Intelligence
• Evaluating Code Reasoning Abilities of Large Language Models Under Real-World Settings
• T5Gemma 2: Seeing, Reading, and Understanding Longer
• IaC Generation with LLMs: An Error Taxonomy and A Study on Configuration Knowledge Injection
• Vector Prism: Animating Vector Graphics by Stratifying Semantic Structure
• OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value
• PerfCoder: Large Language Models for Interpretable Code Performance Optimization
• Sharing State Between Prompts and Programs
• Workflows vs Agents for Code Translation
• EvoLattice: Persistent Internal-Population Evolution through Multi-Alternative Quality-Diversity Graph Representations for LLM-Guided Program Discovery
• CAPE: Capability Achievement via Policy Execution
• ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding
• Scaling Laws for Code: Every Programming Language Matters
• neuralFOMO: Can LLMs Handle Being Second Best? Measuring Envy-Like Preferences in Multi-Agent Settings
• AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning
• Socratic Students: Teaching Language Models to Learn by Asking Questions
• CODE ACROSTIC: Robust Watermarking for Code Generation
• SIGMA: An AI-Empowered Training Stack on Early-Life Hardware
• NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
• Low-Rank Compression of Language Models via Differentiable Rank Selection
• Taint-Based Code Slicing for LLMs-based Malicious NPM Package Detection
• Citation-Grounded Code Comprehension: Preventing LLM Hallucination Through Hybrid Retrieval and Graph-Augmented Context
• Training Versatile Coding Agents in Synthetic Environments
• Market-Bench: Evaluating Large Language Models on Introductory Quantitative Trading and Market Dynamics
• AutoFSM: A Multi-agent Framework for FSM Code Generation with IR and SystemC-Based Testing
• AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference
• A-LAMP: Agentic LLM-Based Framework for Automated MDP Modeling and Policy Generation
• Theoretical Foundations of GPU-Native Compilation for Rapid Code Iteration
• Progress over Points: Reframing LM Benchmarks Around Scientific Objectives
• REMODEL-LLM: Transforming C code to Java using LLMs
• Towards Privacy-Preserving Code Generation: Differentially Private Code Language Models
• Long-horizon Reasoning Agent for Olympiad-Level Mathematical Problem Solving
• PACIFIC: a framework for generating benchmarks to check Precise Automatically Checked Instruction Following In Code
• GLOW: Graph-Language Co-Reasoning for Agentic Workflow Performance Prediction
• ESS: An Offload-Centric Latent-Cache Management Architecture for DeepSeek-V3.2-Exp
• Zero-shot 3D Map Generation with LLM Agents: A Dual-Agent Architecture for Procedural Content Generation
• Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebases
• ATLAS: Automated Toolkit for Large-Scale Verified Code Synthesis
• Design Space Exploration of DMA based Finer-Grain Compute Communication Overlap
• Scaling Behavior of Discrete Diffusion Language Models
• Understanding Chain-of-Thought Effectiveness in Code Generation: An Empirical and Information-Theoretic Analysis
• Chasing Shadows: Pitfalls in LLM Security Research
• LLaDA2.0: Scaling Up Diffusion Language Models to 100B
• Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokens
• Learning Unmasking Policies for Diffusion Language Models
• Revisiting the Scaling Properties of Downstream Metrics in Large Language Model Training
• Multicalibration for LLM-based Code Generation
• Autonomous Issue Resolver: Towards Zero-Touch Code Maintenance
• Towards a Science of Scaling Agent Systems
• Token Sugar: Making Source Code Sweeter for LLMs through Token-Efficient Shorthand
• Secure or Suspect? Investigating Package Hallucinations of Shell Command in Original and Quantized LLMs
• SimpleDevQA: Benchmarking Large Language Models on Development Knowledge QA
• Single-Agent Scaling Fails Multi-Agent Intelligence: Towards Foundation Models with Native Multi-Agent Intelligence
• Bridging Code Graphs and Large Language Models for Better Code Understanding
• ReasonBENCH: Benchmarking the (In)Stability of LLM Reasoning
• PCMind-2.1-Kaiyuan-2B Technical Report
• AutoICE: Automatically Synthesizing Verifiable C Code via LLM-driven Evolution
• Persian-Phi: Efficient Cross-Lingual Adaptation of Compact LLMs via Curriculum Learning
• VulnLLM-R: Specialized Reasoning LLM with Agent Scaffold for Vulnerability Detection
• BabelCoder: Agentic Code Translation with Specification Alignment
• From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs
• CFCEval: Evaluating Security Aspects in Code Generated by Large Language Models
• Beyond Token-level Supervision: Unlocking the Potential of Decoding-based Regression via Reinforcement Learning
• Whatever Remains Must Be True: Filtering Drives Reasoning in LLMs, Shaping Diversity
• PRiSM: An Agentic Multimodal Benchmark for Scientific Reasoning via Python-Grounded Evaluation
• Beyond Prototyping: Autonomous, Enterprise-Grade Frontend Development from Pixel to Production via a Specialized Multi-Agent Framework
• PrivCode: When Code Generation Meets Differential Privacy
• AI &amp; Human Co-Improvement for Safer Co-Superintelligence
• Mitigating Catastrophic Forgetting in Target Language Adaptation of LLMs via Source-Shielded Updates
• Eval Factsheets: A Structured Framework for Documenting AI Evaluations
• Yuan3.0 Ultra: A Trillion-Parameter Enterprise-Oriented MoE LLM
• Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
• RLHFSpec: Breaking the Efficiency Bottleneck in RLHF Training via Adaptive Drafting
• PBFuzz: Agentic Directed Fuzzing for PoV Generation
• Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective
• ADAPT: Learning Task Mixtures for Budget-Constrained Instruction Tuning
• Counting Without Running: Evaluating LLMs' Reasoning About Code Complexity
• Semantic Soft Bootstrapping: Long Context Reasoning in LLMs without Reinforcement Learning
• Quantitative Analysis of Technical Debt and Pattern Violation in Large Language Model Architectures
• Decoding Large Language Diffusion Models with Foreseeing Movement
• Reason-Plan-ReAct: A Reasoner-Planner Supervising a ReAct Executor for Complex Enterprise Tasks
• OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
• Context-Aware Hierarchical Learning: A Two-Step Paradigm towards Safer LLMs
• From FLOPs to Footprints: The Resource Cost of Artificial Intelligence
• DAWZY: A New Addition to AI powered "Human in the Loop" Music Co-creation
• InnoGym: Benchmarking the Innovation Potential of AI Agents
• Enhancing Automated Paper Reproduction via Prompt-Free Collaborative Agents
• CryptoQA: A Large-scale Question-answering Dataset for AI-assisted Cryptography
• Feedback Loops and Code Perturbations in LLM-based Software Engineering: A Case Study on a C-to-Rust Translation System
• Parameter-Efficient Subspace Optimization for LLM Fine-Tuning
• Neural steering vectors reveal dose and exposure-dependent impacts of human-AI relationships
• SynthStrategy: Extracting and Formalizing Latent Strategic Insights from LLMs in Organic Chemistry
• Large Language Models Cannot Reliably Detect Vulnerabilities in JavaScript: The First Systematic Benchmark and Evaluation
• LLM-as-a-Judge for Scalable Test Coverage Evaluation: Accuracy, Operational Reliability, and Cost
• MindFuse: Towards GenAI Explainability in Marketing Strategy Co-Creation
• DrawingBench: Evaluating Spatial Reasoning and UI Interaction Capabilities of Large Language Models through Mouse-Based Drawing Tasks
• TradeTrap: Are LLM-based Trading Agents Truly Reliable and Faithful?
• BackportBench: A Multilingual Benchmark for Automated Backporting of Patches
• CodeDistiller: Automatically Generating Code Libraries for Scientific Coding Agents
• ChartAnchor: Chart Grounding with Structural-Semantic Fidelity
• WaterSearch: A Quality-Aware Search-based Watermarking Framework for Large Language Models
• Bias Injection Attacks on RAG Databases and Sanitization Defenses
• SpeContext: Enabling Efficient Long-context Reasoning with Speculative Context Sparsity in LLMs
• CentaurEval: Benchmarking Human-in-the-Loop Value in Agentic Coding
• ML-Tool-Bench: Tool-Augmented Planning for ML Tasks
• G-KV: Decoding-Time KV Cache Eviction with Global Attention
• Framework-Aware Code Generation with API Knowledge Graph-Constructed Data: A Study on HarmonyOS
• EduEval: A Hierarchical Cognitive Benchmark for Evaluating Large Language Models in Chinese Education
• Trification: A Comprehensive Tree-based Strategy Planner and Structural Verification for Fact-Checking
• Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match
• Generating Verifiable Chain of Thoughts from Exection-Traces
• Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models
• PRISM: Privacy-Aware Routing for Adaptive Cloud-Edge LLM Inference via Semantic Sketch Collaboration
• AI Co-Artist: A LLM-Powered Framework for Interactive GLSL Shader Animation Evolution
• DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving
• TreeCoder: Systematic Exploration and Optimisation of Decoding and Constraints for LLM Code Generation
• Statistical Independence Aware Caching for LLM Workflows
• Decomposed Trust: Exploring Privacy, Adversarial Robustness, Fairness, and Ethics of Low-Rank LLMs
• IntAttention: A Fully Integer Attention Pipeline for Efficient Edge Inference
• Code Refactoring with LLM: A Comprehensive Evaluation With Few-Shot Settings
• Multi-Agent Systems for Dataset Adaptation in Software Engineering: Capabilities, Limitations, and Future Directions
• Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models
• BRIDGE: Building Representations In Domain Guided Program Verification
• From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models
• Aragog: Just-in-Time Model Routing for Scalable Serving of Agentic Workflows
• CafeQ: Calibration-free Quantization via Learned Transformations and Adaptive Rounding
• Hierarchical Evaluation of Software Design Capabilities of Large Language Models of Code
• Supporting Students in Navigating LLM-Generated Insecure Code
• Can Vibe Coding Beat Graduate CS Students? An LLM vs. Human Coding Tournament on Market-driven Strategic Planning
• CDLM: Consistency Diffusion Language Models For Faster Sampling
• DRAFT-RL: Multi-Agent Chain-of-Draft Reasoning for Reinforcement Learning-Enhanced LLMs
• Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios
• NNGPT: Rethinking AutoML with Large Language Models
• CLIMATEAGENT: Multi-Agent Orchestration for Complex Climate Data Science Workflows
• R3A: Reliable RTL Repair Framework with Multi-Agent Fault Localization and Stochastic Tree-of-Thoughts Patch Generation
• RPM-MCTS: Knowledge-Retrieval as Process Reward Model with Monte Carlo Tree Search for Code Generation
• Mosaic Pruning: A Hierarchical Framework for Generalizable Pruning of Mixture-of-Experts Models
• DualGauge: Automated Joint Security-Functionality Benchmarking of Specification-Only Code Generation by LLMs and Coding Agents
• Agint: Agentic Graph Compilation for Software Engineering Agents
• SLMFix: Leveraging Small Language Models for Error Fixing with Reinforcement Learning
• Learning Robust Social Strategies with Large Language Models
• Optimizing LLM Code Suggestions: Feedback-Driven Timing with Lightweight State Bounds
• A Multimodal Conversational Agent for Tabular Data Analysis
• CodeR3: A GenAI-Powered Workflow Repair and Revival Ecosystem
• VecIntrinBench: Benchmarking Cross-Architecture Intrinsic Code Migration for RISC-V Vector
• Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost
• Toward Trustworthy Difficulty Assessments: Large Language Models as Judges in Programming and Synthetic Tasks
• Reasoning With a Star: A Heliophysics Dataset and Benchmark for Agentic Scientific Reasoning
• Evaluating perturbation robustness of generative systems that use COBOL code inputs
• Xmodel-2.5: 1.3B Data-Efficient Reasoning SLM
• A²Flow: Automating Agentic Workflow Generation via Self-Adaptive Abstraction Operators
• Can Large Language Models Reason About Complex Execution Paths? An Empirical Study on Python
• LLM Assisted Coding with Metamorphic Specification Mutation Agent
• FAST: Topology-Aware Frequency-Domain Distribution Matching for Coreset Selection
• WavefrontDiffusion: Dynamic Decoding Schedule for Improved Reasoning
• Masked-and-Reordered Self-Supervision for Reinforcement Learning from Verifiable Rewards
• EngChain: A Symbolic Benchmark for Verifiable Multi-Step Reasoning in Engineering
• E³-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models
• Datacenters in the Desert: Feasibility and Sustainability of LLM Inference in the Middle East
• Asking LLMs to Verify First is Almost Free Lunch
• NALA_MAINZ at BLP-2025 Task 2: A Multi-agent Approach for Bangla Instruction to Python Code Generation
• Pass@k Metric for RLVR: A Diagnostic Tool of Exploration, But Not an Objective
• PSM: Prompt Sensitivity Minimization via LLM-Guided Black-Box Optimization
• On 10x Better Scalability: KV Stores Scale Up KV Cache
• Multi-Agent Code Verification via Information Theory
• InfCode: Adversarial Iterative Refinement of Tests and Patches for Reliable Software Issue Resolution
• CARE: Turning LLMs Into Causal Reasoning Expert
• Global Resolution: Optimal Multi-Draft Speculative Sampling via Convex Minimization
• A Causal Perspective on Measuring, Explaining and Mitigating Smells in LLM-Generated Code
• Parameter Importance-Driven Continual Learning for Foundation Models
• Effective Code Membership Inference for Code Completion Models via Adversarial Prompts
• MermaidSeqBench: An Evaluation Benchmark for LLM-to-Mermaid Sequence Diagram Generation
• Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learning
• Beyond Surface-Level Similarity: Hierarchical Contamination Detection for Synthetic Training Data in Foundation Models
• Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?
• Performance and interpretability analysis of code generation large language models
• Can large language models be a cardinality estimator? An empirical study
• Reducing Hallucinations in LLM-Generated Code via Semantic Triangulation
• Group-Aware Reinforcement Learning for Output Diversity in Large Language Models
• UpBench: A Dynamically Evolving Real-World Labor-Market Agentic Benchmark Framework Built for Human-Centric AI
• Striking the Right Balance between Compute and Copy: Improving LLM Inferencing Under Speculative Decoding
• Defending Unauthorized Model Merging via Dual-Stage Weight Protection
• FarSkip-Collective: Unhobbling Blocking Communication in Mixture of Experts Models
• Virtual Width Networks
• STaR: Towards Cognitive Table Reasoning via Slow-Thinking Large Language Models
• Utilizing LLMs for Industrial Process Automation: A Case Study on Modifying RAPID Programs
• Go-UT-Bench: A Fine-Tuning Dataset for LLM-Based Unit Test Generation in Go
• Towards a Human-in-the-Loop Framework for Reliable Patch Evaluation Using an LLM-as-a-Judge
• InData: Towards Secure Multi-Step, Tool-Based Data Analysis
• When Data is the Algorithm: A Systematic Study and Curation of Preference Optimization Datasets
• HPCAgentTester: A Multi-Agent LLM Approach for Enhanced HPC Unit Test Generation
• ExPairT-LLM: Exact Learning for LLM Code Selection by Pairwise Queries
• AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
• Speculative Decoding in Decentralized LLM Inference: Turning Communication Latency into Computation Throughput
• Reasoning: From Reflection to Solution
• EnchTable: Unified Safety Alignment Transfer in Fine-tuned Large Language Models
• Steering Pretrained Drafters during Speculative Decoding
• CrochetBench: Can Vision-Language Models Move from Describing to Doing in Crochet Domain?
• The Ouroboros of Benchmarking: Reasoning Evaluation in an Era of Saturation
• Evaluating from Benign to Dynamic Adversarial: A Squid Game for Large Language Models
• OSGym: Super-Scalable Distributed Data Engine for Generalizable Computer Agents
• Think-at-Hard: Selective Latent Iterations to Improve Reasoning Language Models
• AlphaResearch: Accelerating New Algorithm Discovery with Language Models
• Why Should the Server Do It All?: A Scalable, Versatile, and Model-Agnostic Framework for Server-Light DNN Inference over Massively Distributed Clients via Training-Free Intermediate Feature Compression
• Investigating The Smells of LLM Generated Code
• GazeCopilot: Evaluating Novel Gaze-Informed Prompting for AI-Supported Code Comprehension and Readability
• A Small Leak Sinks All: Exploring the Transferable Vulnerability of Source Code Models
• MURPHY: Multi-Turn GRPO for Self Correcting Code Generation
• LLM-GROP: Visually Grounded Robot Task and Motion Planning with Large Language Models
• Smart but Costly? Benchmarking LLMs on Functional Accuracy and Energy Efficiency
• Procedural Knowledge Improves Agentic LLM Workflows
• On the Creativity of AI Agents
• RedOne 2.0: Rethinking Domain-specific LLM Post-Training in Social Networking Services
• SemanticForge: Repository-Level Code Generation through Semantic Knowledge Graphs and Constraint Satisfaction
• MobileLLM-Pro Technical Report
• LLM For Loop Invariant Generation and Fixing: How Far Are We?
• Better Datasets Start From RefineLab: Automatic Optimization for High-Quality Dataset Refinement
• You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
• Route Experts by Sequence, not by Token
• Towards Resource-Efficient Multimodal Intelligence: Learned Routing among Specialized Expert Models
• Self-Abstraction from Grounded Experience for Plan-Guided Policy Refinement
• FLEX: Continuous Agent Evolution via Forward Learning from Experience
• An Empirical Study of Reasoning Steps in Thinking Code LLMs
• Catching Contamination Before Generation: Spectral Kill Switches for Agents
• SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?
• A Metamorphic Testing Perspective on Knowledge Distillation for Language Models of Code: Does the Student Deeply Mimic the Teacher?
• Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding
• KLASS: KL-Guided Fast Inference in Masked Diffusion Models
• Motif 2 12.7B technical report
• Dynamic Stability of LLM-Generated Code
• SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models
• PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
• From Model to Breach: Towards Actionable LLM-Generated Vulnerabilities Reporting
• Where Do LLMs Still Struggle? An In-Depth Analysis of Code Generation Benchmarks
• How Natural Language Proficiency Shapes GenAI Code for Software Engineering Tasks
• Benchmarking and Studying the LLM-based Agent System in End-to-End Software Development
• PSD2Code: Automated Front-End Code Generation from Design Files via Multimodal Large Language Models
• TwIST: Rigging the Lottery in Transformers with Independent Subnetwork Training
• EDIT-Bench: Evaluating LLM Abilities to Perform Real-World Instructed Code Edits
• Exploring the Feasibility of End-to-End Large Language Model as a Compiler
• Secure Code Generation at Scale with Reflexion
• Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling
• Understanding Robustness of Model Editing in Code LLMs
• Learning-based Cooperative Robotic Paper Wrapping: A Unified Control Policy with Residual Force Control
• PoCo: Agentic Proof-of-Concept Exploit Generation for Smart Contracts
• VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation
• SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks
• LTD-Bench: Evaluating Large Language Models by Letting Them Draw
• VFocus: Better Verilog Generation from Large Language Model via Focused Reasoning
• FATE: A Formal Benchmark Series for Frontier Algebra of Multiple Difficulty Levels
• Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models
• TapOut: A Bandit-Based Approach to Dynamic Speculative Decoding
• SmartMLOps Studio: Design of an LLM-Integrated IDE with Automated MLOps Pipelines for Model Development and Monitoring
• Context-Guided Decompilation: A Step Towards Re-executability
• Detecting Vulnerabilities from Issue Reports for Internet-of-Things
• HarnessLLM: Automatic Testing Harness Generation via Reinforcement Learning
• DPO-F+: Aligning Code Repair Feedback with Developers' Preferences
• IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation
• A Comprehensive Empirical Evaluation of Agent Frameworks on Code-centric Software Engineering Tasks
• GrowthHacker: Automated Off-Policy Evaluation Optimization Using Code-Modifying LLM Agents
• AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUs
• Can Large Language Models Detect Real-World Android Software Compliance Violations?
• GDPR-Bench-Android: A Benchmark for Evaluating Automated GDPR Compliance Detection in Android
• SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding
• A Hierarchical Imprecise Probability Approach to Reliability Assessment of Large Language Models
• ReMind: Understanding Deductive Code Reasoning in LLMs
• Sherlock: Reliable and Efficient Agentic Workflow Execution
• ARC-GEN: A Mimetic Procedural Benchmark Generator for the Abstraction and Reasoning Corpus
• What a diff makes: automating code migration with large language models
• Culture Cartography: Mapping the Landscape of Cultural Knowledge
• CodeAlignBench: Assessing Code Generation Models on Developer-Preferred Code Adjustments
• TempoBench: Evaluating Temporal Causal Reasoning in Large Language Models
• DeepCompress: A Dual Reward Strategy for Dynamically Exploring and Compressing Reasoning Chains
• Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments
• DRAMA: Unifying Data Retrieval and Analysis for Open-Domain Analytic Queries
• Towards Understanding Self-play for LLM Reasoning
• Inferring multiple helper Dafny assertions with LLMs
• Cross-Platform Evaluation of Reasoning Capabilities in Foundation Models
• LoRAQuant: Mixed-Precision Quantization of LoRA to Ultra-Low Bits
• CATArena: Evaluating Evolutionary Capabilities of Code Agents via Iterative Tournaments
• Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systems
• EdgeRunner 20B: Military Task Parity with GPT-5 while Running on the Edge
• Automated Extract Method Refactoring with Open-Source LLMs: A Comparative Study
• Nexus: Execution-Grounded Multi-Agent Test Oracle Synthesis
• OmniEduBench: A Comprehensive Chinese Benchmark for Evaluating Large Language Models in Education
• BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning
• Do LLMs Signal When They're Right? Evidence from Neuron Agreement
• Test-Time Alignment of LLMs via Sampling-Based Optimal Control in pre-logit space
• Reasoning Curriculum: Bootstrapping Broad LLM Reasoning from Math
• Beyond Synthetic Benchmarks: Evaluating LLM Performance on Real-World Class-Level Code Generation
• Predicate Renaming via Large Language Models
• QCoder Benchmark: Bridging Language Generation and Quantum Hardware through Simulator-Based Feedback
• Generalizing Test-time Compute-optimal Scaling as an Optimizable Graph
• Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents
• User Misconceptions of LLM-Based Conversational Programming Assistants
• SkillGate: Cost Efficient Runtime Malicious Skill File Detection in Coding Agents
• From Role Prompt to Infinite Thinking: Exploiting Persona Conditioning for Inference Cost Attacks in LLMs
• Experiential Reflective Learning for Self-Improving LLM Agents
• From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models
• Fewer Clarifications, Better Code: Benchmarking Cross-Session Personalized Ambiguity Adaptation in Coding Assistants
• AgenticCANN: Automated Ascend C Operator Generation via Knowledge-Augmented Agentic Evolution
• Metis: Memory Foundation Model
• One Run Is Not an Idea: The Implementation Lottery in Automated Research
• Between Gradient and Natural Gradient: A Continuum of LoRA Initializations
• Try Again, Don't Look Back: Blind Resampling Outperforms Self-Repair in Small Code Models
• Large Language Models for Software Engineering Diagrams: A Systematic Review of UML and ER modelling
• Pramana: A Composable, Domain-Specific Backend for Empirical Networking Research
• TraceCoder: Explainable and Auditable Code Generation with Position-Key Snippet Versioning
• Cross-Model Cross-Language AI Coding Agent Performance: Accuracy and Speed of Parallel CLRS Algorithms
• EsoLang-Bench: Evaluating Genuine Reasoning in Large Language Models via Esoteric Programming Languages
• Training Language Models via Neural Cellular Automata
• CodeSpec: Dual Executable Specifications for Agentic Long-Horizon Feature Development
• Will Scaling Improve Social Simulation with LLMs?
• MRCoder: An Efficient Context Selecting Approach for Repository-Level Code Generation
• ReCo: Reweighting GRPO Against Distributional Concentration
• ALMAS: an Autonomous LLM-based Multi-Agent Software Engineering Framework
• RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents
• Two Calls Beat Five Agents: Evaluating Multi-Agent Pipelines Against Self-Refinement for Local Language Models
• Functional Cache Grafting: Robust and Rapid Code-Policy Synthesis for Embodied Agents
• Teaching Machine Learning to Software Engineers
• REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage
• Human-Aligned Code Readability Assessment with Large Language Models
• Fine-Tuning GPT-5 for GPU Kernel Generation
• HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing
• Better Call Grep: Evaluating and Improving Grep-Like Lexical Retrieval for Repository-Level Code Completion
• Understanding the Characteristics of LLM-Generated Property-Based Tests in Exploring Edge Cases
• Large Language Model for Verilog Code Generation: Literature Review and the Road Ahead
• StorageXTuner: An LLM Agent-Driven Automatic Tuning Framework for Heterogeneous Storage Systems
• Pearl: A Foundation Model for Placing Every Atom in the Right Location
• What Limits Agentic Systems Efficiency?
• Parallel Loop Transformer for Efficient Test-Time Computation Scaling
• CodeWiki: Evaluating AI's Ability to Generate Holistic Documentation for Large-Scale Codebases
• Uncovering Gaps Between RFC Updates and TCP/IP Implementations: LLM-Facilitated Differential Checks on Intermediate Representations
• APTBench: Benchmarking Agentic Potential of Base LLMs During Pre-Training
• LLM-as-a-Judge for Software Engineering: Literature Review, Vision, and the Road Ahead
• Lifecycle-Aware code generation: Leveraging Software Engineering Phases in LLMs
• Evaluating the effectiveness of LLM-based interoperability
• Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
• Compiler.next: A Search-Based Compiler to Power the AI-Native Future of Software Engineering
• ScaLoRA: Optimally Scaled Low-Rank Adaptation for Efficient High-Rank Fine-Tuning
• Multi-Agent Evolve: LLM Self-Improve through Co-evolution
• The Best of N Worlds: Aligning Reinforcement Learning with Best-of-N Sampling via max@k Optimisation
• PAHQ: Accelerating Automated Circuit Discovery through Mixed-Precision Inference Optimization
• Increasing LLM Coding Capabilities through Diverse Synthetic Coding Tasks
• A Survey on LLM Mid-Training
• Advantage Shaping as Surrogate Reward Maximization: Unifying Pass@K Policy Gradients
• TALM: Dynamic Tree-Structured Multi-Agent Framework with Long-Term Memory for Scalable Code Generation
• CodeAD: Synthesize Code of Rules for Log-based Anomaly Detection with LLMs
• Is Your Prompt Poisoning Code? Defect Induction Rates and Security Mitigation Strategies
• Agent-GSPO: Communication-Efficient Multi-Agent Systems via Group Sequence Policy Optimization
• PortGPT: Towards Automated Backporting Using Large Language Models
• Harnessing the Power of Large Language Models for Software Testing Education: A Focus on ISTQB Syllabus
• Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs
• Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing
• Software Engineering Agents for Embodied Controller Generation : A Study in Minigrid Environments
• Co-Sight: Enhancing LLM-Based Agents via Conflict-Aware Meta-Verification and Trustworthy Reasoning with Structured Facts
• Risk Management for Mitigating Benchmark Failure Modes: BenchRisk
• Rethinking the Design of Reinforcement Learning-Based Deep Research Agents
• When Models Outthink Their Safety: Mitigating Self-Jailbreak in Large Reasoning Models with Chain-of-Guardrails
• Securing AI Agent Execution
• Model Merging with Functional Dual Anchors
• Beyond Pairwise: Empowering LLM Alignment With Ranked Choice Modeling
• Self-Rewarding PPO: Aligning Large Language Models with Demonstrations Only
• Designing and Evaluating Hint Generation Systems for Science Education
• Attention Sinks in Diffusion Language Models
• Large Language Models for Fault Localization: An Empirical Study
• Relative-Based Scaling Law for Neural Language Models
• Not-a-Bandit: Provably No-Regret Drafter Selection in Speculative Decoding for LLMs
• Data-Centric Lessons To Improve Speech-Language Pretraining
• SmartSwitch: Advancing LLM Reasoning by Overcoming Underthinking via Promoting Deeper Thought Exploration
• Zhyper: Factorized Hypernetworks for Conditioned LLM Fine-Tuning
• LLavaCode: Compressed Code Representations for Retrieval-Augmented Code Generation
• From Large to Small: Transferring CUDA Optimization Expertise via Reasoning Graph
• An Experimental Study of Real-Life LLM-Proposed Performance Improvements
• QiMeng-SALV: Signal-Aware Learning for Verilog Code Generation
• GAPO: Robust Advantage Estimation for Real-World Code LLMs
• Knowledge-Guided Multi-Agent Framework for Application-Level Software Code Generation
• SODBench: A Large Language Model Approach to Documenting Spreadsheet Operations
• DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inference
• No Compute Left Behind: Rethinking Reasoning and Sampling with Masked Diffusion Models
• LLMartini: Seamless and Interactive Leveraging of Multiple LLMs through Comparison and Composition
• LAPRAD: LLM-Assisted PRotocol Attack Discovery
• SheetBrain: A Neuro-Symbolic Agent for Accurate Reasoning over Complex and Large Spreadsheets
• KAT-Coder Technical Report
• Evaluating LLM Story Generation through Large-scale Network Analysis of Social Structures
• From Quarter to All: Accelerating Speculative LLM Decoding via Floating-Point Exponent Remapping and Parameter Sharing
• VAPU: System for Autonomous Legacy Code Modernization
• CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment
• AlphaOPT: Formulating Optimization Programs with Self-Improving LLM Experience Library
• Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs
• RESCUE: Retrieval Augmented Secure Code Generation
• CircuitSeer: Mining High-Quality Data by Probing Mathematical Reasoning Circuits in LLMs
• Learning from Generalization Patterns: An Evaluation-Driven Approach to Enhanced Data Augmentation for Fine-Tuning Small Language Models
• Planned Diffusion
• Any-Depth Alignment: Unlocking Innate Safety Alignment of LLMs to Any-Depth
• AdapTrack: Constrained Decoding without Distorting LLM's Output Intent
• StreamingThinker: Large Language Models Can Think While Reading
• Select-Then-Decompose: From Empirical Analysis to Adaptive Selection Strategy for Task Decomposition in Large Language Models
• Soft-Masked Diffusion Language Models
• Automated Algorithm Design for Auto-Tuning Optimizers
• Verification-Aware Planning for Multi-Agent Systems
• JT-Safe: Intrinsically Enhancing the Safety and Trustworthiness of LLMs
• Integrating Performance Tools in Model Reasoning for GPU Kernel Optimization
• TREAT: A Code LLMs Trustworthiness / Reliability Evaluation and Testing Framework
• Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Model
• Reasoning Distillation and Structural Alignment for Improved Code Generation
• Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning
• QuanBench: Benchmarking Quantum Code Generation with Large Language Models
• Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs
• Learning to Answer from Correct Demonstrations
• Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models
• A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
• Beyond Function-Level Search: Repository-Aware Dual-Encoder Code Retrieval with Adversarial Verification
• Attention Is All You Need for KV Cache in Diffusion LLMs
• Reasoning with Sampling: Your Base Model is Smarter Than You Think
• Programmatic Representation Learning with Language Models
• LLM Agents for Automated Web Vulnerability Reproduction: Are We There Yet?
• Purifying Task Vectors in Knowledge-Aware Subspace for Model Merging
• Code-driven Number Sequence Calculation: Enhancing the inductive Reasoning Abilities of Large Language Models
• Helmsman: Autonomous Synthesis of Federated Learning Systems via Collaborative LLM Agents
• E2Edev: Benchmarking Large Language Models in End-to-End Software Development Task
• ZeroFalse: Improving Precision in Static Analysis with LLMs
• A Systematic Study of Time Limit Exceeded Errors in Online Programming Assignments
• Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction
• UniCode: A Framework for Generating High Quality Competitive Coding Problems
• Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models
• RLSR: Reinforcement Learning with Supervised Reward Outperforms SFT in Instruction Following
• CodeEvolve: An open source evolutionary coding agent for algorithm discovery and optimization
• David vs. Goliath: A comparative study of different-sized LLMs for code generation in the domain of automotive scenario generation
• Breaking Memorization Barriers in LLM Code Fine-Tuning via Information Bottleneck for Improved Generalization
• Training LLM Agents to Empower Humans
• NOSA: Native and Offloadable Sparse Attention
• OpenDerisk: An Industrial Framework for AI-Driven SRE, with Design, Implementation, and Case Studies
• ConsintBench: Evaluating Language Models on Real-World Consumer Intent Understanding
• A Matter of Representation: Towards Graph-Based Abstract Code Generation
• Program of Thoughts for Financial Reasoning: Leveraging Dynamic In-Context Examples and Generative Retrieval
• Evaluating Arabic Large Language Models: A Survey of Benchmarks, Methods, and Gaps
• Attention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization
• Beyond Postconditions: Can Large Language Models infer Formal Contracts for Automatic Software Verification?
• KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems
• Ax-Prover: A Deep Reasoning Agentic Framework for Theorem Proving in Mathematics and Quantum Physics
• Diff-XYZ: A Benchmark for Evaluating Diff Understanding
• MoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Experts
• A Survey on Parallel Reasoning
• Towards Engineering Multi-Agent LLMs: A Protocol-Driven Approach
• Do Large Language Models Respect Contracts? Evaluating and Enforcing Contract-Adherence in Code Generation
• UALM: Unified Audio Language Model for Understanding, Generation and Reasoning
• TopoAlign: A Framework for Aligning Code to Math via Topological Decomposition
• Representation-Based Exploration for Language Models: From Test-Time to Post-Training
• Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Models
• ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding
• TypePilot: Leveraging the Scala Type System for Secure LLM-generated Code
• Defects4C: Benchmarking Large Language Model Repair Capability with C/C++ Bugs
• Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation
• DebugTA: An LLM-Based Agent for Simplifying Debugging and Teaching in Programming Education
• Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief States
• LogiNumSynth: Synthesizing Joint Logical-Numerical Reasoning Problems for Language Models
• GeoVLMath: Enhancing Geometry Reasoning in Vision-Language Models via Cross-Modal Reward for Auxiliary Line Creation
• DND: Boosting Large Language Models with Dynamic Nested Depth
• A Survey on Agentic Multimodal Large Language Models
• Scaling Agents for Computer Use
• APLOT: Robust Reward Modeling via Adaptive Preference Learning with Optimal Transport
• MC#: Mixture Compressor for Mixture-of-Experts Large Models
• Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
• AMiD: Knowledge Distillation for LLMs with α-mixture Assistant Distribution
• Beyond Consensus: Mitigating the Agreeableness Bias in LLM Judge Evaluations
• UpSafe^∘C: Upcycling for Controllable Safety in Large Language Models
• Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization
• ECO: Enhanced Code Optimization via Performance-Aware Prompting for Code-LLMs
• One Token Embedding Is Enough to Deadlock Your Large Reasoning Model
• Testing and Enhancing Multi-Agent Systems for Robust Code Generation
• Rethinking LLM Evaluation: Can We Evaluate LLMs with 200x Less Data?
• BenchPress: A Human-in-the-Loop Annotation System for Rapid Text-to-SQL Benchmark Curation
• DynaSpec: Context-aware Dynamic Speculative Sampling for Large-Vocabulary Language Models
• MatryoshkaThinking: Recursive Test-Time Scaling Enables Efficient Reasoning
• Learning to Guarantee Type Correctness in Code Generation through Type-Guided Program Synthesis
• Failure-Driven Workflow Refinement
• StelLA: Subspace Learning in Low-rank Adaptation using Stiefel Manifold
• Decoupled DiLoCo for Resilient Distributed Pre-training
• On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
• Do Thought Streams Matter? Evaluating Reasoning in Gemini Vision-Language Models for Video Scene Understanding
• Agentic Compilation: Mitigating the LLM Rerun Crisis for Minimized-Inference-Cost Web Automation
• Therefore I am. I Think
• SOL-ExecBench: Speed-of-Light Benchmarking for Real-World GPU Kernels Against Hardware Limits
• LiveOIBench: Can Large Language Models Outperform Human Contestants in Informatics Olympiads?
• StatEval: A Comprehensive Benchmark for Large Language Models in Statistics
• Identifying &amp; Interactively Refining Ambiguous User Goals for Data Visualization Code Generation
• Logit Arithmetic Elicits Long Reasoning Capabilities Without Training
• Context-Aware Visual Prompting: Automating Geospatial Web Dashboards with Large Language Models and Agent Self-Validation for Decision Support
• InteractScience: Programmatic and Visually-Grounded Evaluation of Interactive Scientific Demonstration Code Generation
• A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System
• MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics
• On the Provable Performance Guarantee of Efficient Reasoning Models
• DITING: A Multi-Agent Evaluation Framework for Benchmarking Web Novel Translation
• Scaling Laws for Code: A More Data-Hungry Regime
• CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards
• dInfer: An Efficient Inference Framework for Diffusion Language Models
• RA-Gen: A Controllable Code Generation Framework Using ReAct for Multi-Agent Task Execution
• Guided Star-Shaped Masked Diffusion
• First Try Matters: Revisiting the Role of Reflection in Reasoning Models
• LLMs Learn to Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactions
• CREST-Search: Comprehensive Red-teaming for Evaluating Safety Threats in Large Language Models Powered by Web Search
• Fewer Weights, More Problems: A Practical Attack on LLM Pruning
• Upfront Chain-of-Thought: A Cooperative Framework for Chain-of-Thought Compression
• Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMs
• Robust Heuristic Algorithm Design with LLMs
• Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimization
• Beyond Pass@k: Breadth-Depth Metrics for Reasoning Boundaries
• MOSAIC: Multi-agent Orchestration for Task-Intelligent Scientific Coding
• FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Experts
• Learning on the Job: An Experience-Driven Self-Evolving Agent for Long-Horizon Tasks
• What Is Your Agent's GPA? A Framework for Evaluating Agent Goal-Plan-Action Alignment
• Vibe Coding: Toward an AI-Native Paradigm for Semantic and Intent-Driven Programming
• Fortifying LLM-Based Code Generation with Graph-Based Reasoning on Secure Coding Practices
• Don't Adapt Small Language Models for Tools; Adapt Tool Schemas to the Models
• Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Dense
• More Data or Better Data? A Critical Analysis of Data Selection and Synthesis for Mathematical Reasoning
• Accelerating Diffusion LLM Inference via Local Determinism Propagation
• SHANKS: Simultaneous Hearing and Thinking for Spoken Language Models
• AMAS: Adaptively Determining Communication Topology for LLM-based Multi-Agent System
• SoftMatcha 2: A Fast and Soft Pattern Matcher for Trillion-Scale Corpora
• On Randomness in Agentic Evals
• Towards Interpretable and Inference-Optimal COT Reasoning with Sparse Autoencoder-Guided Generation
• Mid-Training of Large Language Models: A Survey
• Beyond Models: A Framework for Contextual and Cultural Intelligence in African AI Deployment
• GRACE: A Language Model Framework for Explainable Inverse Reinforcement Learning
• What MLLMs Learn about When they Learn about Multimodal Reasoning
• Flipping the Dialogue: Training and Evaluating User Language Models
• POME: Post Optimization Model Edit via Muon-style Projection
• TGPR: Tree-Guided Policy Refinement for Robust Self-Debugging of LLMs
• Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levels
• Code Semantic Zooming
• Off-Trajectory Reasoning: Can LLMs Collaborate on Reasoning Trajectory?
• RECODE-H: A Benchmark for Research Code Development with Interactive Human Feedback
• SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
• CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credits
• lm-Meter: Unveiling Runtime Inference Latency for On-Device Language Models
• Influence Functions for Efficient Data Selection in Reasoning
• ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Models
• EEPO: Exploration-Enhanced Policy Optimization via Sample-Then-Forget
• Mellum: Production-Grade in-IDE Contextual Code Completion with Multi-File Project Understanding
• Fine-Tuning Masked Diffusion for Provable Self-Correction
• Vul-R2: A Reasoning LLM for Automated Vulnerability Repair
• AMAQ: Adaptive Mixed-bit Activation Quantization for Collaborative Parameter Efficient Fine-tuning
• DeepV: A Model-Agnostic Retrieval-Augmented Framework for Verilog Code Generation with a High-Quality Knowledge Base
• Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models
• Slm-mux: Orchestrating small language models for reasoning
• SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs
• RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems
• Modeling Student Learning with 3.8 Million Program Traces
• FreshBrew: A Benchmark for Evaluating AI Agents on Java Code Migration
• ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMs
• FedSRD: Sparsify-Reconstruct-Decompose for Communication-Efficient Federated Large Language Models Fine-Tuning
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches
• Context Length Alone Hurts LLM Performance Despite Perfect Retrieval
• LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning
• Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
• Exploring the Power of Diffusion Large Language Models for Software Engineering: An Empirical Investigation
• GRACE: Generative Representation Learning via Contrastive Policy Optimization
• AutoEmpirical: LLM-Based Automated Research for Empirical Software Fault Analysis
• Scaling Sequence-to-Sequence Generative Neural Rendering
• GA4GC: Greener Agent for Greener Code via Multi-Objective Configuration Optimization
• SPOGW: a Score-based Preference Optimization method via Group-Wise comparison for workflows
• Bamboo: LLM-Driven Discovery of API-Permission Mappings in the Android Framework
• The Debate on RLVR Reasoning Capability Boundary: Shrinkage, Expansion, or Both? A Two-Stage Dynamic View
• Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models
• Multi Language Models for On-the-Fly Syntax Highlighting
• MacroBench: A Novel Testbed for Web Automation Scripts via Large Language Models
• RESTRAIN: From Spurious Votes to Signals -- Self-Driven RL with Self-Penalization
• Don't Pass@k: A Bayesian Framework for Large Language Model Evaluation
• What Shapes a Creative Machine Mind? Comprehensively Benchmarking Creativity in Foundation Models
• Refactoring with LLMs: Bridging Human Expertise and Machine Understanding
• When Names Disappear: Revealing What LLMs Actually Understand About Code
• Rainbow Padding: Mitigating Early Termination in Instruction-Tuned Diffusion LLMs
• Token Hidden Reward: Steering Exploration-Exploitation in Group Relative Deep Reinforcement Learning
• REFINE: Enhancing Program Repair Agents through Context-Aware Patch Refinement
• Backdoor-Powered Prompt Injection Attacks Nullify Defense Methods
• Unlocking Reasoning Capabilities in LLMs via Reinforcement Learning Exploration
• GuidedSampling: Steering LLMs Towards Diverse Candidate Solutions at Inference-Time
• PLSemanticsBench: Large Language Models As Programming Language Interpreters
• GramTrans: A Better Code Representation Approach in Code Generation
• AutoMaAS: Self-Evolving Multi-Agent Architecture Search for Large Language Models
• AgenticRAG: Tool-Augmented Foundation Models for Zero-Shot Explainable Recommender Systems
• LLM Agents for Automated Dependency Upgrades
• Model-Agnostic Correctness Assessment for LLM-Generated Code via Dynamic Internal Representation Selection
• TravelBench : Exploring LLM Performance in Low-Resource Domains
• Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks
• mR3: Multilingual Rubric-Agnostic Reward Reasoning Models
• Safety Instincts: LLMs Learn to Trust Their Internal Compass for Self-Defense
• Uncovering the Computational Ingredients of Human-Like Representations in LLMs
• Opal: A Modular Framework for Optimizing Performance using Analytics and LLMs
• Can Emulating Semantic Translation Help LLMs with Code Translation? A Study Based on Pseudocode
• Color2Struct: efficient and accurate deep-learning inverse design of structural color with controllable inference
• Characterizing Model Behavior Under Synthetic Data Training: An Empirical Study Across Scales and Mixing Ratios
• LongCodeZip: Compress Long Context for Code Language Models
• TokMem: Tokenized Procedural Memory for Large Language Models
• Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerability
• A-VERT: Agnostic Verification with Embedding Ranking Targets
• CodeChemist: Functional Knowledge Transfer for Low-Resource Code Generation via Test-Time Scaling
• Stochastic Self-Organization in Multi-Agent Systems
• Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling
• HiSpec: Hierarchical Speculative Decoding for LLMs
• Multi-LLM Orchestration for High-Quality Code Generation: Exploiting Complementary Model Strengths
• MAVUL: Multi-Agent Vulnerability Detection via Contextual Reasoning and Interactive Refinement
• Towards Ecologically Valid LLM Benchmarks: Understanding and Designing Domain-Centered Evaluations for Journalism Practitioners
• Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models
• LoRAFusion: Efficient LoRA Fine-Tuning for LLMs
• Train Large, Deploy Compact: Structured Compression for Compact Low-Rank Adaptation
• Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmark
• Can LLMs Write Mathematics Papers? A Case Study in Reservoir Computing
• Revealing the Power of Post-Training for Small Language Models via Knowledge Distillation
• dParallel: Learnable Parallel Decoding for dLLMs
• AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Size
• Communication-Efficient and Accurate Approach for Aggregation in Federated Low-Rank Adaptation
• Clip-Low Increases Entropy and Clip-High Decreases Entropy in Reinforcement Learning of Large Language Models
• DyFlow: Dynamic Workflow Framework for Agentic Reasoning
• Accelerating LLM Inference with Precomputed Query Storage
• Lita: Light Agent Uncovers the Agentic Coding Capabilities of LLMs
• Overthinking Reduction with Decoupled Rewards and Curriculum Data Scheduling
• Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reuse
• Expert Merging: Model Merging with Unsupervised Expert Alignment and Importance-Guided Layer Chunking
• Learning to Reason as Action Abstractions with Scalable Mid-Training RL
• Thinking Sparks!: Emergent Attention Heads in Reasoning Models During Post Training
• RFG: Test-Time Scaling for Diffusion Large Language Model Reasoning with Reward-Free Guidance
• MixtureVitae: Open Web-Scale Pretraining Dataset With High Quality Instruction and Reasoning Data Built from Permissive-First Text Sources
• BloomAPR: A Bloom's Taxonomy-based Framework for Assessing the Capabilities of LLM-Powered APR Solutions
• Adaptive Test-Time Reasoning via Reward-Guided Dual-Phase Search
• UniAPL: A Unified Adversarial Preference Learning Framework for Instruct-Following
• Towards Reliable Generation of Executable Workflows by Foundation Models
• Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning
• Random Policy Valuation is Enough for LLM Reasoning with Verifiable Rewards
• Agentic Exploration of Physics Models
• MobileLLM-R1: Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes
• Evaluating SAP Joule for Code Generation
• SeaPO: Strategic Error Amplification for Robust Preference Optimization of Large Language Models
• Bridging Developer Instructions and Code Completion Through Instruction-Aware Fill-in-the-Middle Paradigm
• SemGuard: Real-Time Semantic Evaluator for Correcting LLM-Generated Code
• Towards Safe Reasoning in Large Reasoning Models via Corrective Intervention
• MAS²: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models
• Risk-Sensitive RL for Alleviating Exploration Dilemmas in Large Language Models
• ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Models
• TENET: Leveraging Tests Beyond Validation for Code Generation
• InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptation
• Why Tree-Style Branching Matters for Thought Advantage Estimation in GRPO
• Automatically Generating Web Applications from Requirements Via Multi-Agent Test-Driven Development
• Advancements in Generative AI: A Comprehensive Review of GANs, GPT, Autoencoders, Diffusion Model, and Transformers
• ARS: Adaptive Reasoning Suppression for Efficient Large Reasoning Language Models
• Short window attention enables long-term memorization
• Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding
• PerfBench: Can Agents Resolve Real-World Performance Bugs?
• Sequential Diffusion Language Models
• Future-Proofing Programmers: Optimal Knowledge Tracing for AI-Assisted Personalized Education
• LLM/Agent-as-Data-Analyst: A Survey
• Toward Preference-aligned Large Language Models via Residual-based Model Steering
• HiPO: Hybrid Policy Optimization for Dynamic Reasoning in LLMs
• Beyond Benchmarks: Understanding Mixture-of-Experts Models through Internal Mechanisms
• Anchored Supervised Fine-Tuning
• Demystifying the Lifecycle of Failures in Platform-Orchestrated Agentic Workflows
• PAT-Agent: Autoformalization for Model Checking
• Fast Thinking for Large Language Models
• Timber: Training-free Instruct Model Refining with Base via Effective Rank
• Pretraining Scaling Laws for Generative Evaluations of Language Models
• SolContractEval: A Benchmark for Evaluating Contract-Level Solidity Code Generation
• EAPO: Enhancing Policy Optimization with On-Demand Expert Assistance
• Artificial Intelligence-Powered Assessment Framework for Skill-Oriented Engineering Lab Education
• RANGER -- Repository-Level Agent for Graph-Enhanced Retrieval
• From text to traits: exploring the role of large language models in plant breeding
• Planner Aware Path Learning in Diffusion Language Models Training
• A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Models
• The Matthew Effect of AI Programming Assistants: A Hidden Bias in Software Evolution
• Understanding and Enhancing the Planning Capability of Language Models via Multi-Token Prediction
• Test-Time Policy Adaptation for Enhanced Multi-Turn Interactions with LLMs
• SysMoBench: Evaluating AI on Formally Modeling Complex Real-World Systems
• Effective Quantization of Muon Optimizer States
• Multiplayer Nash Preference Optimization
• CATMark: A Context-Aware Thresholding Framework for Robust Cross-Task Watermarking in Large Language Models
• Local Success Does Not Compose: Benchmarking Large Language Models for Compositional Formal Verification
• Tracing the Representation Geometry of Language Models from Pretraining to Post-training
• CoDA: Coding LM via Diffusion Adaptation
• Protocode: Prototype-Driven Interpretability for Code Generation in LLMs
• Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models
• BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software
• d²Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
• Critique-Coder: Enhancing Coder Models by Critique Reinforcement Learning
• What Do They Fix? LLM-Aided Categorization of Security Patches for Critical Memory Bugs
• Dynamic Experts Search: Enhancing Reasoning in Mixture-of-Experts LLMs at Test Time
• The Emergence of Altruism in Large-Language-Model Agents Society
• A model of errors in transformers
• Stochastic activations
• MultiMat: Multimodal Program Synthesis for Procedural Materials using Large Multimodal Models
• FeatBench: Evaluating Coding Agents on Feature Implementation for Vibe Coding
• Bridging Draft Policy Misalignment: Group Tree Optimization for Speculative Decoding
• Multi-Agent Path Finding via Offline RL and LLM Collaboration
• SK2Decompile: LLM-based Two-Phase Binary Decompilation from Skeleton to Skin
• The Rogue Scalpel: Activation Steering Compromises LLM Safety
• Reinforcement Learning-Guided Chain-of-Draft for Token-Efficient Code Generation
• GSM-Agent: Understanding Agentic Reasoning Using Controllable Environments
• A Benchmark for Localizing Code and Non-Code Issues in Software Projects
• Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
• AgentPack: A Dataset of Code Changes, Co-Authored by Agents and Humans
• QoNext: Towards Next-generation QoE for Foundation Models
• RobustFlow: Towards Robust Agentic Workflow Generation
• Compiling by Proving: Language-Agnostic Automatic Optimization from Formal Semantics
• FastGRPO: Accelerating Policy Optimization via Concurrency-aware Speculative Decoding and Online Draft Learning
• PSRT: Accelerating LRM-based Guard Models via Prefilled Safe Reasoning Traces
• An LLM-Powered Agent for Real-Time Analysis of the Vietnamese IT Job Market
• VibeCodeHPC: An Agent-Based Iterative Prompting Auto-Tuner for HPC Code Generation Using LLMs
• Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Data
• Blockwise Hadamard high-Rank Adaptation for Parameter-Efficient LLM Fine-Tuning
• Towards Transparent AI: A Survey on Explainable Language Models
• InvBench: Can LLMs Accelerate Program Verification with Invariant Synthesis?
• A State-of-the-Art SQL Reasoning Model using RLVR
• Automotive-ENV: Benchmarking Multimodal Agents in Vehicle Interface Systems
• Expanding Reasoning Potential in Foundation Model by Learning Diverse Chains of Thought Patterns
• Predicting LLM Reasoning Performance with Small Proxy Model
• StyleBench: Evaluating thinking styles in Large Language Models
• Verification Limits Code LLM Training
• SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs
• TyphoonMLA: A Mixed Naive-Absorb MLA Kernel For Shared Prefix
• Mixture of Thoughts: Learning to Aggregate What Experts Think, Not Just What They Say
• RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs
• d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation
• Enhancing Python Programming Education with an AI-Powered Code Helper: Design, Implementation, and Impact
• Thinking Augmented Pre-training
• Automated Multi-Agent Workflows for RTL Design
• Evaluating and Mitigating Errors in LLM-Generated Web API Integrations
• V-GameGym: Visual Game Generation for Code Large Language Models
• GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow
• FastEagle: Cascaded Drafting for Accelerating Speculative Decoding
• Beyond Language Barriers: Multi-Agent Coordination for Multi-Language Code Generation
• SpecMamba: Accelerating Mamba Inference on FPGA with Speculative Decoding
• Enhancing Linear Attention with Residual Learning
• Intuition to Evidence: Measuring AI's True Impact on Developer Productivity
• Nano Bio-Agents (NBA): Small Language Model Agents for Genomics
• SCOPE: Synthetic Conditional Objectives for Policy Evolution in Black-Box Combinatorial Optimization
• Can Large Language Models Resolve Real Java Merge Conflicts? An Evaluation with a Calibrated LLM-as-Judge
• Σ-Mem: An Online Reliability Memory for LLM-based Multi-Agent Systems
• Specification-Guided Synthesis of Deadlock-Free Communication Protocol Refinements with Large Language Models
• Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents
• DataClawEval: A Benchmark for Data Engineering Agents in Real Industrial Harness
• LoRA Scaffolded Policy Optimization (LSPO): A Sampling-Time Low-Rank Scaffold for Recovering Reinforcement-Learning Gradient on Zero-Reward Cliff Prompts
• Beyond Rephrasing: Book-Level Organization Improves Synthetic Textbook Data for Mid-Training
• A Policy-Driven Runtime Layer for Agentic LLM Serving
• Beyond KV Reconstruction: Functional Reconstruction for MLA Draft Models in Speculative Decoding
• SWE-NFI: Studying and Benchmarking Coding Agents for Non-Functional Improvements
• KernelGenBench: A Multi-Source and Multi-Chip Benchmark for LLM-based Kernel Generation
• Multi-Head Attention Residuals
• Configuration Smells in AGENTS.md Files: Common Mistakes in Configuring Coding Agents
• The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence
• Large Language Models for Accessible Reporting of Bioinformatics Analyses in Interdisciplinary Contexts
• Can neural networks do arithmetic? A survey on the elementary numerical skills of state-of-the-art deep learning models
• RELISH: LLM REgression with a Latent Iterative State Head
• Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs
• Benchmark for Assessing Olfactory Perception of Large Language Models
• DFlash: Block Diffusion for Flash Speculative Decoding
• Language Models Coupled with Metacognition Can Outperform Reasoning Models
• Reinforcement Learning on Pre-Training Data
• AgentInit: Initializing LLM-based Multi-Agent Systems via Diversity and Expertise Orchestration for Effective and Efficient Collaboration
• When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models
• LLM-Enhanced Self-Evolving Reinforcement Learning for Multi-Step E-Commerce Payment Fraud Risk Detection
• Prior-based Noisy Text Data Filtering: Fast and Strong Alternative For Perplexity
• NGRPO: Negative-enhanced Group Relative Policy Optimization
• Code Driven Planning with Domain-Adaptive Critic
• Introducing LongCat-Flash-Thinking: A Technical Report
• Symphony-MoE: Harmonizing Disparate Pre-trained Models into a Coherent Mixture-of-Experts
• Actions Speak Louder than Prompts: A Large-Scale Study of LLMs for Graph Inference
• Speculate Deep and Accurate: Lossless and Training-Free Acceleration for Offloaded LLMs via Substitute Speculative Decoding
• Reasoning Core: A Scalable RL Environment for LLM Symbolic Reasoning
• Program Synthesis via Test-Time Transduction
• Mitigating Strategy-Selection Bias in Reasoning for More Effective Test-Time Scaling
• MapCoder-Lite: Distilling Multi-Agent Coding into a Single Small LLM
• Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphs
• Probabilistic Token Alignment for Large Language Model Fusion
• PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
• SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
• From Prediction to Understanding: Will AI Foundation Models Transform Brain Science?
• MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE
• ACCeLLiuM: Supervised Fine-Tuning for Automated OpenACC Pragma Generation
• Improving User Interface Generation Models from Designer Feedback
• Evaluating LLM Generated Detection Rules in Cybersecurity
• Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Models
• The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software Development
• Shift Parallelism: Low-Latency, High-Throughput LLM Inference for Dynamic Workloads
• Pipeline Parallelism is All You Need for Optimized Early-Exit Based Self-Speculative Decoding
• Generating High-Quality Datasets for Code Editing via Open-Source Language Models
• CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics
• Overhearing LLM Agents: A Survey, Taxonomy, and Roadmap
• Analyzing and Mitigating Surface Bias in Code Evaluation Metrics
• LNE-Blocking: An Efficient Framework for Contamination Mitigation Evaluation on Large Language Models
• RulER: Automated Rule-Based Semantic Error Localization and Repair for Code Translation
• FURINA: Free from Unmergeable Router via LINear Aggregation of mixed experts
• CARGO: A Framework for Confidence-Aware Routing of Large Language Models
• Evaluating the Effectiveness of Coverage-Guided Fuzzing for Testing Deep Learning Library APIs
• Automating Modelica Module Generation Using Large Language Models: A Case Study on Building Control Description Language
• Self-Improvement of Language Models by Post-Training on Multi-Agent Debate
• Orion: Fuzzing Workflow Automation
• Masked Diffusion Models as Energy Minimization
• THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoning
• Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearning
• A Study on Thinking Patterns of Large Reasoning Models in Code Generation
• Aegis: Automated Error Generation and Attribution for Multi-Agent Systems
• CLMTracing: Black-box User-level Watermarking for Code Language Model Tracing
• Prompt Stability in Code LLMs: Measuring Sensitivity across Emotion- and Personality-Driven Variations
• VerilogMonkey: Exploring Parallel Scaling for Automated Verilog Code Generation with LLMs
• CodeLSI: Leveraging Foundation Models for Automated Code Generation with Low-Rank Optimization and Domain-Specific Instruction Tuning
• Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning
• GitHub's Copilot Code Review: Can AI Spot Security Flaws Before You Commit?
• DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
• Crash Report Enhancement with Large Language Models: An Empirical Study
• Single-stream Policy Optimization
• Automating Code Generation for Semiconductor Equipment Control from Developer Utterances with LLMs
• SCoGen: Scenario-Centric Graph-Based Synthesis of Real-World Code Problems
• From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
• Don't Change My View: Ideological Bias Auditing in Large Language Models
• A Systematic Evaluation of Parameter-Efficient Fine-Tuning Methods for the Security of Code LLMs
• WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning
• Root Cause Analysis of Radiation Oncology Incidents Using Large Language Models
• Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issues
• WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents
• Selective Risk Certification for LLM Outputs via Information-Lift Statistics: PAC-Bayes, Robustness, and Skeleton Design
• Evaluating Large Language Models for Functional and Maintainable Code in Industrial Settings: A Case Study at ASML
• A New Benchmark for Evaluating Code Translation with Third-Party Libraries
• Reasoned Safety Alignment: Ensuring Jailbreak Defense via Answer-Then-Check
• POT: Inducing Overthinking in LLMs via Black-Box Iterative Optimization
• From Legacy Fortran to Portable Kokkos: An Autonomous Agentic AI Workflow
• From Evaluation to Enhancement: Large Language Models for Zero-Knowledge Proof Code Generation
• UniPar: A Unified LLM-Based Framework for Parallel and Accelerated Code Translation in HPC
• Good Vibrations? A Qualitative Study of Co-Creation, Communication, Flow, and Trust in Vibe Coding
• Semantic Fusion with Fuzzy-Membership Features for Controllable Language Modelling
• Harnessing Optimization Dynamics for Curvature-Informed Model Merging
• Rethinking Technology Stack Selection with AI Coding Proficiency
• Difficulty-Aware Agentic Orchestration for Query-Specific Multi-Agent Workflows
• Beyond Autoregression: An Empirical Study of Diffusion Large Language Models for Code Generation
• PersonaX: Multimodal Datasets with LLM-Inferred Behavior Traits
• ViScratch: Using Large Language Models and Gameplay Videos for Automated Feedback in Scratch
• Judge Q: Trainable Queries for Optimized Information Retention in KV Cache Eviction
• SciML Agents: Write the Solver, Not the Solution
• Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning
• ZapGPT: Free-form Language Prompting for Simulated Cellular Control
• Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems
• Combating the Memory Walls: Optimization Pathways for Long-Context Agentic LLM Inference
• Compass-v3: Scaling Domain-Specific LLMs for Multilingual E-Commerce in Southeast Asia
• TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla
• BRoverbs -- Measuring how much LLMs understand Portuguese proverbs
• SWE-Mirror: Scaling Issue-Resolving Datasets by Mirroring Issues Across Repositories
• Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
• AutoVeriFix: Automatically Correcting Errors and Enhancing Functional Correctness in LLM-Generated Verilog Code
• Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
• ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation
• SCoder: Iterative Self-Distillation for Bootstrapping Small-Scale Data Synthesizers to Empower Code LLMs
• Language Self-Play For Data-Free Training
• WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
• Astra: A Multi-Agent System for GPU Kernel Performance Optimization
• GitHub Copilot AI pair programmer: Asset or Liability?
• Unleashing the True Potential of LLMs: A Feedback-Triggered Self-Correction with Long-Term Multipath Decoding
• Beyond Two-Stage Training: Cooperative SFT and RL for LLM Reasoning
• Learning to Synthesize Programs as Interpretable and Generalizable\n Policies
• LAMDAS: LLM as an Implicit Classifier for Domain-specific Data Selection
• Efficiently Ranking Software Variants with Minimal Benchmarks
• Ban&amp;Pick: Ehancing Performance and Efficiency of MoE-LLMs via Smarter Routing
• Rethinking Reasoning Quality in Large Language Models through Enhanced Chain-of-Thought via RL
• Stack Overflow Is Not Dead Yet: Crowd Answers Still Matter
• Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms
• GRACE: Graph-Guided Repository-Aware Code Completion through Hierarchical Code Fusion
• Hyperbolic Large Language Models
• Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts
• Towards a Unified View of Large Language Model Post-Training
• RL's Razor: Why Online Reinforcement Learning Forgets Less
• How Small is Enough? Empirical Evidence of Quantized Small Language Models for Automated Program Repair
• On Robustness and Reliability of Benchmark-Based Evaluation of LLMs
• Learning to Deliberate: Meta-policy Collaboration for Agentic LLMs with Multi-agent Reinforcement Learning
• CelloAI: Leveraging Large Language Models for HPC Software Development in High Energy Physics
• A Probabilistic Inference Scaling Theory for LLM Self-Correction
• Emergent Hierarchical Reasoning in LLMs through Reinforcement Learning
• app.build: A Production Framework for Scaling Agentic Prompt-to-App Generation with Environment Scaffolding
• AetherCode: Evaluating LLMs' Ability to Win In Premier Programming Competitions
• Mixture-of-Clustered-Experts: Advancing Expert Specialization and Generalization in Instruction Tuning
• LLM Agents for Generating Microservice-based Applications: how complex is your specification?
• Implicit Reasoning in Large Language Models: A Comprehensive Survey
• ReCode: Improving LLM-based Code Repair with Fine-Grained Retrieval-Augmented Generation
• GridMind: LLMs-Powered Agents for Power System Analysis and Operations
• Jointly Reinforcing Diversity and Quality in Language Model Generations
• Automated Repair of C Programs Using Large Language Models
• The Fools are Certain; the Wise are Doubtful: Exploring LLM Confidence in Code Completion
• GradeSQL: Test-Time Inference with Outcome Reward Models for Text-to-SQL Generation from Large Language Models
• Beyond the Surface: A Solution-Aware Retrieval Model for Competition-level Code Generation
• Dream-Coder 7B: An Open Diffusion Language Model for Code
• Throttling Web Agents Using Reasoning Gates
• DSDE: Dynamic Speculative Decoding with KLD Stability for Real-World Serving
• LongCat-Flash Technical Report
• Can Large Language Models Master Complex Card Games?
• From CVE Entries to Verifiable Exploits: An Automated Multi-Agent Framework for Reproducing CVEs
• Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMs
• Inductive Biases and Variable Creation in Self-Attention Mechanisms
• LLM-Generated Explanations Do Not Suffice for Ultra-Strong Machine Learning
• Analysis of Error Sources in LLM-based Hypothesis Search for Few-Shot Rule Induction
• The Gold Medals in an Empty Room: Diagnosing Metalinguistic Reasoning in LLMs with Camlang
• SHERPA: A Model-Driven Framework for Large Language Model Execution
• Comparative Analysis of Large Language Models for the Machine-Assisted Resolution of User Intentions
• RepoMark: A Data-Usage Auditing Framework for Code Large Language Models
• PDTrim: Targeted Pruning for Prefill-Decode Disaggregation in Inference
• Human-Written vs. AI-Generated Code: A Large-Scale Study of Defects, Vulnerabilities, and Complexity
• Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning
• BLUEX Revisited: Enhancing Benchmark Coverage with Automatic Captioning
• Re4: Scientific Computing Agent with Rewriting, Resolution, Review and Revision
• AI Agentic Vulnerability Injection And Transformation with Optimized Reasoning
• UI-Bench: A Benchmark for Evaluating Design Capabilities of AI Text-to-App Tools
• Lethe: Purifying Backdoored Large Language Models with Knowledge Dilution
• Multi-Agent Penetration Testing AI for the Web
• Quantum Verifiable Rewards for Post-Training Qiskit Code Assistant
• A Systematic Review on the Generative AI Applications in Human Medical Genomics
• Diffusion Language Models Know the Answer Before Decoding
• SoK: Large Language Model Copyright Auditing via Fingerprinting
• Benchmarking Hindi LLMs: A New Suite of Datasets and a Comparative Analysis
• Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
• ReST-RL: Achieving Accurate Code Reasoning of LLMs with Optimized Self-Training and Decoding
• Functional Consistency of LLM Code Embeddings: A Self-Evolving Data Synthesis Framework for Benchmarking
• Alignment with Fill-In-the-Middle for Enhancing Code Generation
• Learning Game-Playing Agents with Generative Code Optimization
• Athena: Intermediate Representations for Iterative Scaffolded App Generation with an LLM
• LongReasonArena: A Long Reasoning Benchmark for Large Language Models
• Predicting the Order of Upcoming Tokens Improves Language Modeling
• SynthCoder: A Synthetical Strategy to Tune LLMs for Code Completion
• Understanding Tool-Integrated Reasoning
• An Investigation on Group Query Hallucination Attacks
• GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging
• Enabling MoE on the Edge via Importance-Driven Expert Scheduling
• Dynamic Collaboration of Multi-Language Models based on Minimal Complete Semantic Units
• UltraMemV2: Memory Networks Scaling to 120B Parameters with Superior Long-Context Learning
• Toward Edge General Intelligence with Agentic AI and Agentification: Concepts, Technologies, and Future Directions
• Optimal Sparsity of Mixture-of-Experts Language Models for Reasoning Tasks
• Beyond Benchmark: LLMs Evaluation with an Anthropomorphic and Value-oriented Roadmap
• QAgent: An LLM-based Multi-Agent System for Autonomous OpenQASM programming
• Language Models For Generalised PDDL Planning: Synthesising Sound and Programmatic Policies
• Cognitive Agents Powered by Large Language Models for Agile Software Project Management
• VERIRL: Boosting the LLM-based Verilog Code Generation via Reinforcement Learning
• Latent Self-Consistency for Reliable Majority-Set Selection in Short- and Long-Answer Reasoning
• Training Language Model Agents to Find Vulnerabilities with CTF-Dojo
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
• WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling
• A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code
• VocabTailor: Dynamic Vocabulary Selection for Downstream Tasks in Small Language Models
• Dream 7B: Diffusion Large Language Models
• An Empirical Study of Knowledge Distillation for Code Understanding Tasks
• Fine-tuning and prompt engineering for large language models-based code review automation
• Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset
• Choose Your Programming Copilot: A Comparison of the Program Synthesis Performance of GitHub Copilot and Genetic Programming
• MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers
• Static Analysis as a Feedback Loop: Enhancing LLM-Generated Code Beyond Correctness
• ZPD-SCA: Unveiling the Blind Spots of LLMs in Assessing Students' Cognitive Abilities
• Assessing the Quality and Security of AI-Generated Code: A Quantitative Analysis
• NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model
• Your Reward Function for RL is Your Best PRM for Search: Unifying RL and Search-Based TTS
• MultiFuzz: A Dense Retrieval-based Multi-Agent System for Network Protocol Fuzzing
• Measuring LLM Code Generation Stability via Structural Entropy
• ChronoLLM: Customizing Language Models for Physics-Based Simulation Code Generation
• Prompt Orchestration Markup Language
• COMPASS: A Multi-Dimensional Benchmark for Evaluating Code Generation in Large Language Models
• Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Exploration
• Generics and Default Reasoning in Large Language Models
• COCO: Cognitive Operating System with Continuous Oversight for Multi-Agent Workflow Reliability
• Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR
• DPad: Efficient Diffusion Language Models with Suffix Dropout
• G²RPO-A: Guided Group Relative Policy Optimization with Adaptive Guidance
• PC-Sampler: Position-Aware Calibration of Decoding Bias in Masked Diffusion Models
• Ambiguity Resolution with Human Feedback for Code Writing Tasks
• RAJ-PGA: Reasoning-Activated Jailbreak and Principle-Guided Alignment Framework for Large Reasoning Models
• Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation
• Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction
• Help or Hurdle? Rethinking Model Context Protocol-Augmented Large Language Models
• Is GPT-OSS Good? A Comprehensive Evaluation of OpenAI's Latest Open Source Models
• "My productivity is boosted, but ..." Demystifying Users' Perception on AI Coding Assistants
• The Self-Execution Benchmark: Measuring LLMs' Attempts to Overcome Their Lack of Self-Execution
• Where to Start Alignment? Diffusion Large Language Model May Demand a Distinct Position
• Uncovering Systematic Failures of LLMs in Verifying Code Against Natural Language Specifications
• DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
• Benchmarking LLM-based Agents for Single-cell Omics Analysis
• Optimizing Token Choice for Code Watermarking: An RL Approach
• RPKT: Learning What You Don't -- Know Recursive Prerequisite Knowledge Tracing in Conversational AI Tutors for Personalized Learning
• Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models
• AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities
• Inclusion Arena: An Open Platform for Evaluating Large Foundation Models with Real-World Apps
• Hallucination in LLM-Based Code Generation: An Automotive Case Study
• RAG for Geoscience: What We Expect, Gaps and Opportunities
• Diffusion is a code repair operator and generator
• Hell or High Water: Evaluating Agentic Recovery from External Failures
• Data and Context Matter: Towards Generalizing AI-based Software Vulnerability Detection
• Large Model Empowered Embodied AI: A Survey on Decision-Making and Embodied Learning
• Benchmark Dataset Generation and Evaluation for Excel Formula Repair with LLMs
• Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
• Constrained Decoding of Diffusion LLMs with Context-Free Grammars
• Amazon Nova AI Challenge -- Trusted AI: Advancing secure, AI-assisted software development
• VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models
• Next Edit Prediction: Learning to Predict Code Edits from Context and Interaction History
• SaraCoder: Orchestrating Semantic and Structural Cues for Resource-Optimized Repository-Level Code Completion
• Improving Diversity in Language Models: When Temperature Fails, Change the Loss
• ReqInOne: A Large Language Model-Based Agent for Software Requirements Specification Generation
• EvoCurr: Self-evolving Curriculum with Behavior Code Generation for Complex Decision-making
• Your Coding Intent is Secretly in the Context and You Should Deliberately Infer It Before Completion
• LaajMeter: A Framework for LaaJ Evaluation
• CodeGrad: Integrating Multi-Step Verification with Gradient-Based LLM Refinement
• Teaching Code Refactoring Using LLMs
• READER: Retrieval-Assisted Drafter for Efficient LLM Inference
• Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments
• Special-Character Adversarial Attacks on Open-Source Language Model
• InternBootcamp Technical Report: Boosting LLM Reasoning with Verifiable Task Scaling
• Retrospective Sparse Attention for Efficient Long-Context Generation
• Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models
• From Natural Language to Solver-Ready Power System Optimization: An LLM-Assisted, Validation-in-the-Loop Framework
• AdaptFlow: Adaptive Workflow Optimization via Meta-Learning
• Evaluating Large Language Models as Expert Annotators
• From Trial-and-Error to Improvement: A Systematic Analysis of LLM Exploration Mechanisms in RLVR
• GVGAI-LLM: Evaluating Large Language Model Agents with Infinite Games
• Expert Preference-based Evaluation of Automated Related Work Generation
• Tailored Emotional LLM-Supporter: Enhancing Cultural Sensitivity
• Urbanite: A Dataflow-Based Framework for Human-AI Interactive Alignment in Urban Visual Analytics
• AutoAssert 1: A LoRA Fine-Tuned LLM Model for Efficient Automated Assertion Generation
• Dynamic Benchmark Construction for Evaluating Large Language Models on Real-World Codes
• Schema Lineage Extraction at Scale: Multilingual Pipelines, Composite Evaluation, and Language-Model Benchmarks
• Confidence Estimation for Text-to-SQL in Large Language Models
• Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection
• Devstral: Fine-tuning Language Models for Coding Agent Applications
• Optimizing Prompt Sequences using Monte Carlo Tree Search for LLM-Based Optimization
• Bifrost-1: Bridging Multimodal LLMs and Diffusion Models with Patch-level CLIP Latents
• SKATE, a Scalable Tournament Eval: Weaker LLMs differentiate between stronger ones using verifiable challenges
• Pruning the Unsurprising: Efficient LLM Reasoning via First-Token Surprisal
• Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
• RTTC: Reward-Guided Collaborative Test-Time Compute
• WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent
• On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification
• Embedding Alignment in Code Generation for Audio
• Understanding and Mitigating Errors of LLM-Generated RTL Code
• MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
• CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RL
• Klear-CodeTest: Scalable Test Case Generation for Code Reinforcement Learning
• Can Large Language Models Integrate Spatial Data? Empirical Insights into Reasoning Strengths and Computational Weaknesses
• Why Attention Fails: A Taxonomy of Faults in Attention-Based Neural Networks
• IFDECORATOR: Wrapping Instruction Following Reinforcement Learning with Verifiable Rewards
• FinMMR: Make Financial Numerical Reasoning More Multimodal, Comprehensive, and Challenging
• CARD: A Cache-Assisted Parallel Speculative Decoding Framework via Query-and-Correct Paradigm for Accelerating LLM Inference
• Forgetting: A New Mechanism Towards Better Large Language Model Fine-tuning
• StackPilot: Autonomous Function Agents for Scalable and Environment-Free Code Execution
• ShoppingBench: A Real-World Intent-Grounded Shopping Benchmark for LLM-based Agents
• WARP -- Web-Augmented Real-time Program Repairer: A Real-Time Compilation Error Resolution using LLMs and Web-Augmented Synthesis
• Experimental Analysis of Productive Interaction Strategy with ChatGPT: User Study on Function and Project-level Code Generation Tasks
• Unveiling Over-Memorization in Finetuning LLMs for Reasoning Tasks
• Tensorized Clustered LoRA Merging for Multi-Task Interference
• GP and LLMs for Program Synthesis: No Clear Winners
• Analyzing Prominent LLMs: An Empirical Study of Performance and Complexity in Solving LeetCode Problems
• More Than a Score: Probing the Impact of Prompt Specificity on LLM Code Generation
• A DbC Inspired Neurosymbolic Layer for Trustworthy Agent Design
• Refining Critical Thinking in LLM Code Generation: A Faulty Premise-based Evaluation Framework
• LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinking
• GUI-ReRank: Enhancing GUI Retrieval with Multi-Modal LLM-based Reranking
• GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Control
• RCP-Merging: Merging Long Chain-of-Thought Models with Domain-Specific Models by Considering Reasoning Capability as Prior
• RegMean++: Enhancing Effectiveness and Generalization of Regression Mean for Model Merging
• Survey of Large Language Models in Extended Reality: Technical Paradigms and Application Frontiers
• A Rolling Stone Gathers No Moss: Adaptive Policy Optimization for Stable Self-Evaluation in Large Multimodal Models
• CTTS: Collective Test-Time Scaling
• Data Dependency-Aware Code Generation from Enhanced UML Sequence Diagrams
• Robot builds a robot's brain: AI generated drone command and control station hosted in the sky
• Polymath: A Self-Optimizing Agent with Dynamic Hierarchical Workflow
• Can LLMs Generate High-Quality Task-Specific Conversations?
• PentestJudge: Judging Agent Behavior Against Operational Requirements
• OptiHive: Ensemble Selection for LLM-Based Optimization via Statistical Modeling
• An Efficient and Adaptive Next Edit Suggestion Framework with Zero Human Instructions in IDEs
• Everyone Contributes! Incentivizing Strategic Cooperation in Multi-LLM Systems via Sequential Public Goods Games
• Decomposing the Entropy-Performance Exchange: The Missing Keys to Unlocking Effective Reinforcement Learning
• Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving
• Meta-RAG on Large Codebases Using Code Summarization
• Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
• MicroMix: Efficient Mixed-Precision Quantization with Microscaling Formats for Large Language Models
• MLP Memory: A Retriever-Pretrained Memory for Large Language Models
• EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
• T-GRAG: A Dynamic GraphRAG Framework for Resolving Temporal Conflicts and Redundancy in Knowledge Retrieval
• TreeDiff: AST-Guided Code Generation with Diffusion LLMs
• How Far Are LLMs from Symbolic Planners? An NLP-Based Perspective
• Importance Sampling is All You Need: Predict LLM's performance on new benchmark by reusing existing benchmark
• Categorical Construction of Logically Verifiable Neural Architectures
• Exploring Direct Instruction and Summary-Mediated Prompting in LLM-Assisted Code Modification
• Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models
• R1-ACT: Efficient Reasoning Model Safety Alignment by Activating Safety Knowledge
• WMAS: A Multi-Agent System Towards Intelligent and Customized Wireless Networks
• Oedipus and the Sphinx: Benchmarking and Improving Visual Language Models for Complex Graphic Reasoning
• Llama-3.1-FoundationAI-SecurityLLM-8B-Instruct Technical Report
• Is LLM-Generated Code More Maintainable &amp; Reliable than Human-Written Code?
• RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization
• Unveiling Super Experts in Mixture-of-Experts Large Language Models
• AutoBridge: Automating Smart Device Integration with Centralized Platform
• DynaSwarm: Dynamically Graph Structure Selection for LLM-based Multi-agent System
• DSBC : Data Science task Benchmarking with Context engineering
• On LLM-Assisted Generation of Smart Contracts from Business Processes
• ChatVis: Large Language Model Agent for Generating Scientific Visualizations
• SMART-Editor: A Multi-Agent Framework for Human-Like Design Editing with Structural Integrity
• GPT-4.1 Sets the Standard in Automated Experiment Design Using Novel Python Libraries
• IFEvalCode: Controlled Code Generation
• From Articles to Code: On-Demand Generation of Core Algorithms from Scientific Publications
• ChemDFM-R: A Chemical Reasoning LLM Enhanced with Atomized Chemical Knowledge

Discussions

• Evaluating Large Language Models Trained on Code [hn, 12 points, 1 comments]
• Evaluating Large Language Models Trained on Code [hn, 11 points, 1 comments]
• Neben der Bildgenerierung ist Codegenerierung eines der KI-Themen, bei denen ich nur Oberflächenwissen habe. Zeitmangel, Interesse, Prioritäten. Aber diese Studie ist relevant. Zwar von 2021, aber vie [bsky, 9 points, 0 comments]
• Evaluating Large Language Models Trained on Code (paper about GH copilot model) [hn, 4 points, 1 comments]
• Evaluating Large Language Models Trained on Code(GitHub Copilot) [hn, 3 points, 0 comments]
• This pair of papers from 2021 laid the foundation for what's possible now and their benchmarks are still standard: arxiv.org/abs/2107.03374 arxiv.org/abs/2108.07732 Their empirical work is exciting, [bsky, 2 points, 1 comments]
• Evaluating Large Language Models Trained on Code [hn, 2 points, 0 comments]
• Evaluating Large Language Models Trained on Code [lobsters, 1 points, 1 comments]
• 🎧 EP027: From Creative Writer to Logic Engine 📄 Codex 🔗 https://arxiv.org/abs/2107.03374 🟢 https://podcasters.spotify.com/pod/show/yun-wu/episodes/EP027-From-Creative-Writer-to-Logic-Engine-e3fkh1 [bsky, 0 points, 0 comments]

Related

• MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation
• Exploring the Robustness of Large Language Models for Solving Programming Problems
• A Systematic Evaluation of Large Language Models of Code
• Persistent memory for AI coding agents: a pre-registered SWE-bench Verified benchmark
• The Program Testing Ability of Large Language Models for Code