OpenHands: An Open Platform for AI Software Developers as Generalist Agents
2024/07/23 by Xingyao Wang, Wang, Xingyao, Boxuan Li +47 · 3 voices · 417 citations
Computer Science · #Biology #Computer science #Ecology #Generalist and specialist species #Multi-Agent Systems and Negotiation #Open platform #Operating system #Software #Software engineering #cs.AI #cs.CL #cs.SE
paper · pdf · doi:10.48550/arxiv.2407.16741
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2024/07/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Abstract
Software is one of the most powerful tools that we humans have at our disposal; it allows a skilled programmer to interact with the world in complex and profound ways. At the same time, thanks to improvements in large language models (LLMs), there has also been a rapid development in AI agents that interact with and affect change in their surrounding environments. In this paper, we introduce OpenHands (f.k.a. OpenDevin), a platform for the development of powerful and flexible AI agents that interact with the world in similar ways to those of a human developer: by writing code, interacting with a command line, and browsing the web. We describe how the platform allows for the implementation of new agents, safe interaction with sandboxed environments for code execution, coordination between multiple agents, and incorporation of evaluation benchmarks. Based on our currently incorporated benchmarks, we perform an evaluation of agents over 15 challenging tasks, including software engineering (e.g., SWE-BENCH) and web browsing (e.g., WEBARENA), among others. Released under the permissive MIT license, OpenHands is a community project spanning academia and industry with more than 2.1K contributions from over 188 contributors.
Cited by
- AuditRepairBench: A Paired-Execution Trace Corpus for Evaluator-Channel Ranking Instability in Agent Repair
- Beyond Text-to-SQL: Can LLMs Really Debug Enterprise ETL SQL?
- Replication Package (v2): Output Format × Model Identity Interaction Effects
- FedAgentKE: Federated Semantic Knowledge Evolution for Heterogeneous Agents
- PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization
- Know Your Agent: Reconnaissance-Driven Pentesting of AI Agents
- Beyond Fail-to-Pass: Iterative Hardening of Co-Generated Bug Reproduction Tests and Fixes
- PhoenixRepair: Rethinking Repair Strategy Exploration in Software Agents
- NVIDIA-labs OO Agents: Native Python Object-Oriented Agents
- IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests
- FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications
- AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
- Knowledge-Centric Self-Improvement
- SWE-Milestone: Evaluating AI Agents on Continuous Software Evolution
- LLM-Based Invariant Testing for Software Functional Bugs
- Athena-Brain Technical Report: An Efficient Robot Brain for General Intelligence and Embodied Interaction
- CodeRescue: Budget-Calibrated Recovery Routing for Coding Agents
- RoboHarness: Memory-Driven Orchestration of Heterogeneous Robot Policies for Long-Horizon Planning
- When and How Context Rot Appears in Coding Agents: A White-Box Study of Agent Skills in Code Auditing
- DepRepair: LLM-Based Source-Code Repair for Dependency Breaking Changes
- Don't Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent Quality
- TRIM: Reducing AI-Generated CodeSlop via Agent Trajectory Minimization
- SWE-Pruner Pro: The Coder LLM Already Knows What to Prune
- Agentic Vulnerability Reasoning on COTS Binaries
- Beyond Resolved Rate: A Non-Functional Quality Study
- Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models
- IssueExec: A Test-Driven Approach for Localizing Software Engineering Issues
- Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering
- Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios
- Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields
- Pomona: Continuous Code Quality Improvement via Small, Agentic Pull Requests at Bloomberg
- Nonuniformity Principle in Human-AI Coworking
- Large-Scale Terminal Agentic Trajectory Generation from Dockerized Environments
- NEMO: Execution-Aware Optimization Modeling via Autonomous Coding Agents
- Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?
- Latent Programming Horizons in Coding Agents
- Self-Harness: Harnesses That Improve Themselves
- Continual Harness: Online Adaptation for Self-Improving Foundation Agents
- Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses
- Constraint Decay: The Fragility of LLM Agents in Backend Code Generation
- How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks
- Dive into Claude Code: The Design Space of Today's and Future AI Agent Systems
- AutoSurrogate: An LLM-driven multi-agent framework for autonomous construction of deep learning surrogate models in subsurface flow
- Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?
- Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem
- Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine
- AISSISTANT: Human-AI Collaborative Review and Perspective Research Workflows in Data Science
- GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
- Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforce
- Gradient-Based Program Repair: Fixing Bugs in Continuous Program Spaces
- Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation
- Agentic Code Reasoning
- NERFIFY: A Multi-Agent Framework for Turning NeRF Papers into Code
- GraphLocator: Graph-guided Causal Reasoning for Issue Localization
- Agentic Software Issue Resolution with Large Language Models: A Survey
- From Cognitive Architectures to Language Agents: A Mechanism-Level Review of Lineage, Convergence, and Migration Gaps
- HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following
- Matryoshka Agent: Unfolding Sub-Agents for Long-Horizon Machine Learning Engineering
- How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Study
- Agent Team Work Zone: An Automated, Persistent Workspace for Long-Lived Claude Code Agent Teams
- Context as a Tool: Context Management for Long-Horizon SWE-Agents
- JETO-Bench: A Reproducible Benchmark for Execution Time Improvement Patches in Java
- TrajAudit: Automated Failure Diagnosis for Agentic Coding Systems
- SF-AMS: Strategic Forgetting for Structured Memory in LLM Agent
- SWE-RM: Execution-free Feedback For Software Engineering Agents
- C2LLM Technical Report: A New Frontier in Code Retrieval via Adaptive Cross-Attention Pooling
- One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents
- ABBEL: Learning Natural-Language Belief States for Memory-Efficient Interaction
- MemEvolve: Meta-Evolution of Agent Memory Systems
- SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios
- Imitation Learning for Multi-turn LM Agents via On-policy Expert Corrections
- NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
- Training Versatile Coding Agents in Synthetic Environments
- Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebases
- SWEnergy: An Empirical Study on Energy Efficiency in Agentic Issue Resolution Frameworks with SLMs
- Autonomous Issue Resolver: Towards Zero-Touch Code Maintenance
- Reformulate, Retrieve, Localize: Agents for Repository-Level Bug Localization
- Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction
- DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle
- InnoGym: Benchmarking the Innovation Potential of AI Agents
- Process-Centric Analysis of Agentic Software Systems
- Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks
- Simple Agents Outperform Experts in Biomedical Imaging Workflow Optimization
- LeechHijack: Covert Computational Resource Exploitation in Intelligent Agent Systems
- BackportBench: A Multilingual Benchmark for Automated Backporting of Patches
- SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agents
- LLM Assisted Coding with Metamorphic Specification Mutation Agent
- SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agent
- InfCode-C++: Intent-Guided Semantic Retrieval and AST-Structured Search for C++ Issue Resolution
- InfCode: Adversarial Iterative Refinement of Tests and Patches for Reliable Software Issue Resolution
- Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning
- Rethinking Kernel Program Repair: Benchmarking and Enhancing LLMs with RGym
- Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?
- Beyond Accuracy: Behavioral Dynamics of Agentic Multi-Hunk Repair
- AlphaResearch: Accelerating New Algorithm Discovery with Language Models
- MURPHY: Feedback-Aware GRPO with Retrospective Credit Assignment for Multi-Turn Code Generation
- Self-Abstraction from Grounded Experience for Plan-Guided Policy Refinement
- Klear-AgentForge: Forging Agentic Intelligence through Posttraining Scaling
- SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?
- SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models
- ArchPilot: A Proxy-Guided Multi-Agent Approach for Machine Learning Engineering
- Collaborative Agents for Automated Program Repair in Ruby
- The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents
- SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks
- Training Proactive and Personalized LLM Agents
- AgentGit: A Version Control Framework for Reliable and Scalable LLM-Powered Multi-Agent Systems
- Interaction as Intelligence Part II: Asynchronous Human-Agent Rollout for Long-Horizon Task Training
- Understanding Code Agent Behaviour: An Empirical Study of Success and Failure Trajectories
- Gistify! Codebase-Level Understanding via Runtime Execution
- Empowering RepoQA-Agent based on Reinforcement Learning Driven by Monte-carlo Tree Search
- The FM Agent
- QCoder Benchmark: Bridging Language Generation and Quantum Hardware through Simulator-Based Feedback
- Completion ≠ Collaboration: Scaling Collaborative Effort with Agents
- Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents
- How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair
- OrchBench: Evaluating Multi-Agent Orchestration Plans in Isolation via Deterministic Simulation
- Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation
- BM25 Wins at Scale: A Scaling Study of Retrieval-Augmented Generation Paradigms
- ExplainBench: Evaluating Code Explanations from Agents
- DREvo: Distilling Recalibrated Historical Experience for Harness Self-Evolution
- Rethinking Self-Evolution: A Constrained Exploration-Exploitation Process for Mitigating Skill Overfitting
- CodeSpec: Dual Executable Specifications for Agentic Long-Horizon Feature Development
- ALMAS: an Autonomous LLM-based Multi-Agent Software Engineering Framework
- Agents' Last Exam
- Nautilus: From One Prompt to Plug-and-Play Robot Learning
- CooperBench: Why Coding Agents Cannot be Your Teammates Yet
- Sharpness-Guided Group Relative Policy Optimization via Probability Shaping
- Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agents
- Towards AI as Colleagues: Multi-Agent System Improves Structured Professional Ideation
- TDFlow: Agentic Workflows for Test Driven Software Engineering
- Alita-G: Self-Evolving Generative Agent for Agent Generation
- Scalable Supervising Software Agents with Patch Reasoner
- How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations
- TOM-SWE: User Mental Modeling For Software Engineering Agents
- Automated Cloud Infrastructure-as-Code Reconciliation with AI Agents
- Code-enabled language models can outperform reasoning models on diverse tasks
- BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills
- TheMCPCompany: Creating General-purpose Agents with Task-specific Tools
- KAT-Coder Technical Report
- EvoSyn: Generalizable Evolutionary Data Synthesis for Verifiable Learning
- SIADAFIX: issue description response for adaptive program repair
- LLM Agents for Automated Web Vulnerability Reproduction: Are We There Yet?
- E2Edev: Benchmarking Large Language Models in End-to-End Software Development Task
- When "Correct" Is Not Safe: Can We Trust Functionally Correct Patches Generated by Code Agents?
- Benchmarking Correctness and Security in Multi-Turn Code Generation
- Lingxi: Repository-Level Issue Resolution Framework Enhanced by Procedural Knowledge Guided Scaling
- Scaling Long-Horizon LLM Agent via Context-Folding
- BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions
- On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters
- MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation
- Effective Strategies for Asynchronous Software Engineering Agents
- How can we assess human-agent interactions? Case studies in software agent design
- FlowSearch: Advancing deep research with dynamic structured knowledge flow
- Past, Present, and Future of Bug Tracking in the Generative AI Era
- Test-Time Reasoners Are Strategic Multiple-Choice Test-Takers
- Learning on the Job: An Experience-Driven Self-Evolving Agent for Long-Horizon Tasks
- AutoMLGen: Navigating Fine-Grained Optimization for Coding Agents
- CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation
- FreshBrew: A Benchmark for Evaluating AI Agents on Java Code Migration
- SpikingMamba: Towards Energy-Efficient Large Language Models via Knowledge Distillation from Mamba
- Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches
- BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Tasks
- REFINE: Enhancing Program Repair Agents through Context-Aware Patch Refinement
- LLM Agents for Automated Dependency Upgrades
- Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks
- Improving Code Localization with Repository Memory
- Lita: Light Agent Uncovers the Agentic Coding Capabilities of LLMs
- BloomAPR: A Bloom's Taxonomy-based Framework for Assessing the Capabilities of LLM-Powered APR Solutions
- Towards Reliable Generation of Executable Workflows by Foundation Models
- Agentic Services Computing
- TENET: Leveraging Tests Beyond Validation for Code Generation
- PerfBench: Can Agents Resolve Real-World Performance Bugs?
- MCPMark: A Benchmark for Stress-Testing Realistic and Comprehensive MCP Use
- HFuzzer: Testing Large Language Models for Package Hallucinations via Phrase-based Fuzzing
- Improving the Efficiency of LLM Agent Systems through Trajectory Reduction
- WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning
- EPO: Entropy-regularized Policy Optimization for LLM Agents Reinforcement Learning
- SecureAgentBench: Benchmarking Secure Code Generation under Realistic Vulnerability Scenarios
- A Benchmark for Localizing Code and Non-Code Issues in Software Projects
- Efficient On-Device Agents via Adaptive Context Management
- An Empirical Study of Coordination Mode as the First-Class Citizen in From-Scratch Multi-Agent Coding
- FAVA: Formal Authorization for Verified Agents with Evidence-Backed Permission Graphs
- RLPF: Reinforcement Learning from Performance Feedback for Code Generation
- SWE-NFI: Studying and Benchmarking Coding Agents for Non-Functional Improvements
- OwlPath: Lossless Knowledge Compression for LLM Bug Repair
- Evaluating LLM Agents on Automated Software Analysis Tasks
- Generalizable End-to-End Tool-Use RL with Synthetic CodeGym
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
- RPG: A Repository Planning Graph for Unified and Scalable Codebase Generation
- MatchFixAgent: Language-Agnostic Autonomous Repository-Level Code Translation Validation and Repair
- OpenLens AI: Fully Autonomous Research Agent for Health Infomatics
- SWE-QA: Can Language Models Answer Repository-level Code Questions?
- Who is Introducing the Failure? Automatically Attributing Failures of Multi-Agent Systems via Spectrum Analysis
- An Empirical Study on Failures in Automated Issue Solving
- Agentic AI for Software: thoughts from Software Engineering community
- Building Coding Agents via Entropy-Enhanced Multi-Turn Preference Optimization
- CodeCureAgent: Automatic Classification and Repair of Static Analysis Warnings
- SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraints
- EnvX: Agentize Everything with Agentic AI
- Agentic Software Engineering: Foundational Pillars and a Research Roadmap
- AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?
- app.build: A Production Framework for Scaling Agentic Prompt-to-App Generation with Environment Scaffolding
- LLM Agents for Generating Microservice-based Applications: how complex is your specification?
- VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use
- Reinforcement Learning for Machine Learning Engineering Agents
- Reflective Paper-to-Code Reproduction Enabled by Fine-Grained Verification
- The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management
- ASIC-Agent: An Autonomous Multi-Agent System for ASIC Design with Benchmark Evaluation
- AI and Agile Software Development: A Research Roadmap from the XP2025 Workshop
- Reliable Weak-to-Strong Monitoring of LLM Agents
- GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging
- MedResearcher-R1: Expert-Level Medical Deep Researcher via A Knowledge-Informed Trajectory Synthesis Framework
- ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents
- Illuminating LLM Coding Agents: Visual Analytics for Deeper Understanding and Enhancement
- LinkAnchor: An Autonomous LLM-Based Agent for Issue-to-Commit Link Recovery
- You Don't Know Until You Click:Automated GUI Testing for Production-Ready Software Evaluation
- AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities
- Position: Intelligent Coding Systems Should Write Programs with Justifications
- AutoIAD: Manager-Driven Multi-Agent Collaboration for Automated Industrial Anomaly Detection
- Empowering Time Series Forecasting with LLM-Agents
- Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning
- Tool-integrated Reinforcement Learning for Repo Deep Search
- Polymath: A Self-Optimizing Agent with Dynamic Hierarchical Workflow
- A Multi-Agent System for Complex Reasoning in Radiology Visual Question Answering
- A Survey on AgentOps: Categorization, Challenges, and Future Directions
- RepoForge: Training a SOTA Fast-thinking SWE Agent with an End-to-End Data Curation Pipeline Synergizing SFT and RL at Scale
- Towards Explorative IRBL: Combining Semantic Retrieval with LLM-driven Iterative Code Exploration
- Multi-Agent Game Generation and Evaluation via Audio-Visual Recordings
- Measuring Harmfulness of Computer-Using Agents
- Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling
- SWE-Exp: Experience-Driven Software Issue Resolution
- SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution
- AutoCodeSherpa: Symbolic Explanations in AI Coding Agents
- ASPERA: A Simulated Environment to Evaluate Planning for Complex Action Execution
- Can Language Models Discover Scaling Laws?
- FaultLine: Automated Proof-of-Vulnerability Generation Using LLM Agents
- Prometheus: Towards Long-Horizon Codebase Navigation for Repository-Level Problem Solving
- Agentic Program Repair from Test Failures at Scale: A Neuro-symbolic approach with static analysis and test execution feedback
- Doc2Feat-Bench: Evaluating Documentation-Driven Feature Addition
- Token Reduction Is Not Cost Reduction
- AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks?
- PassNet: Scaling Large Language Models for Graph Compiler Pass Generation
- DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback
- FutureSim: Replaying World Events to Evaluate Adaptive Agents
- From Skills to Talent: Organising Heterogeneous Agents as a Real-World Company
- PivotRL: High Accuracy Agentic Post-Training at Low Compute Cost
- LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces
- OmniCode: A Benchmark for Evaluating Software Engineering Agents
- Agyn: A Multi-Agent System for Team-Based Autonomous Software Engineering
- LightAutoDS-Tab: Multi-AutoML Agentic System for Tabular Data
- AssertFlip: Reproducing Bugs via Inversion of LLM-Generated Passing Tests
- Aime: Towards Fully-Autonomous Multi-Agent Framework
- ALE-Bench: A Benchmark for Long-Horizon Objective-Driven Algorithm Engineering
- EDINET-Bench: Evaluating LLMs on Complex Financial Tasks using Japanese Financial Statements
- When Developer Aid Becomes Security Debt: A Systematic Analysis of Insecure Behaviors in LLM Coding Agents
- SPICE: An Automated SWE-Bench Labeling Pipeline for Issue Clarity, Test Coverage, and Effort Estimation
- Understanding Automated Program Repair Agents Through the Lens of Traceability: An Empirical Study
- SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner
- Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows
- AgenticRepair: Multi-Faceted Program Context Engineering for Agentic Vulnerability Repair
- StarDojo: Benchmarking Open-Ended Behaviors of Agentic Multimodal LLMs in Production-Living Simulations with Stardew Valley
- Preventing Premature Commitment in Coding Agents with an Evidence-Conditioned Execution Layer
- EasyBCI Agent: Towards Universal Neural Data Preprocessing for Brain-Computer Interfaces
- Agentic-R1: Distilled Dual-Strategy Reasoning
- OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety
- Critique of World Model
- Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions
- Looks Right, Works Right: A Project-Level Benchmark for Multi-Screen Mobile App Generation
- PSearch: Search-based Patch Generation in the Era of LLM-based Automated Program Repair
- How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench
- Hierarchical Knowledge Injection for Improving LLM-based Program Repair
- A Survey of LLM-based Automated Program Repair: Taxonomies, Design Paradigms, and Applications
- RExBench: Can coding agents autonomously implement AI research extensions?
- CoMind: Towards Community-Driven Agents for Machine Learning Engineering
- Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs
- Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories
- SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications
- Is Your Automated Software Engineer Trustworthy?
- Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Fixing
- Context manipulation attacks : Web agents are susceptible to corrupted memory
- Unified Software Engineering Agent as AI Software Engineer
- Towards Pervasive Distributed Agentic Generative AI -- A State of The Art
- Code Reasoning for Software Engineering Tasks: A Survey and A Call to Action
- MCTS-Refined CoT: High-Quality Fine-Tuning Data for LLM-Based Repository Issue Resolution
- Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safety
- Why AI Agents Still Need You: Findings from Developer-Agent Collaborations in the Wild
- The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason
- SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks
- Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards
- EXPEREPAIR: Dual-Memory Enhanced LLM-based Repository-Level Program Repair
- SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks
- CompilerGPT: Leveraging Large Language Models for Analyzing and Acting on Compiler Optimization Reports
- OmniGIRL: A Multilingual and Multimodal Benchmark for GitHub Issue Resolution
- SweRank: Software Issue Localization with Code Ranking
- LLM Code Customization with Visual Results: A Benchmark on TikZ
- AOSpec: Action and Observation Co-Speculation for Low-Latency Agent Serving
- CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent System
- Seed-Coder: Let the Code Model Curate Data for Itself
- Computational Thinking Reasoning in Large Language Models
- Coding Agents with Multimodal Browsing are Generalist Problem Solvers
- CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale
- Dyna-Think: Synergizing Reasoning, Acting, and World Model Simulation in AI Agents
- EXP-Bench: Can AI Conduct AI Research Experiments?
- A Red Teaming Roadmap Towards System-Level Safety
- GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents
- Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering
- WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch
- MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement
- Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers
- CXXCrafter: An LLM-Based Agent for Automated C/C++ Open Source Software Building
- RepoMaster: Autonomous Exploration and Understanding of GitHub Repositories for Complex Task Solving
- CodeMirage: A Multi-Lingual Benchmark for Detecting AI-Generated and Paraphrased Source Code from Production-Level LLMs
- Code Researcher: Deep Research Agent for Large Systems Code and Commit History
- Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution
- MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research
- Co-PatcheR: Collaborative Software Patching with Component(s)-specific Small Reasoning Models
- HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses
- Distilling LLM Agent into Small Models with Retrieval and Code Tools
- Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks
- SWE-Dev: Evaluating and Training Autonomous Feature-Driven Software Development
- Fetch-then-Explore: Decoupling Selection from Extraction over a Persistent Workspace for Search Agents
- VulnGym: Benchmarking Coding Agents for Repository-Level Vulnerability Detection
- LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks
- Long-Horizon Autonomous Architecture Research with a Language-Model Agent: A Behavioural Case Study
- Bole: Efficient Tree Speculation for Hybrid-Attention Language Models
- R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Science
- TimeSeriesGym: A Scalable Benchmark for (Time Series) Machine Learning Engineering Agents
- MADE: Belief-Driven Dual-Agent Coordination for Autonomous Model Deployment
- Vul4Py: Benchmarking Automated Vulnerability Repair in Python with Paired Exploit and Functional Oracles
- InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer Interaction
- Turning Interaction History into Execution State: A Runtime Layer for Long-Horizon Coding Agents
- LoopsBench: From Harness Engineering to Loop Engineering in Benchmarking Coding Agent
- From Monoliths to Swarms: A Study of Attack Surface Evolution in the Transition to Multi-Agent Web Systems
- MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering
- SVRepair: Structured Visual Reasoning for Automated Program Repair
- Terminal Agents Suffice for Enterprise Automation
- AIRA2: Overcoming Bottlenecks in AI Research Agents
- MobileDev-Bench: A Benchmark for Issue Resolution in Mobile Application Development
- EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments
- Toward Generalist Autonomous Research via Hypothesis-Tree Refinement
- From Translation to Superset: Benchmark-Driven Evolution of a Production AI Agent from Rust to Python
- Group-Evolving Agents: Open-Ended Self-Improvement via Experience Sharing
- VeriTrace: Human-Like Temporal Exploration Completes Agentic Action Space
- When Policies Change Probabilities: Modular Decision-Making for LLM Code Review
- MDArena: Evaluating Coding Agents on Realistic Molecular Dynamics Workflows
- EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments
- Large Databases Need Small, Open-Weight Language Models
- ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development
- SciNav: A General Agent Framework for Scientific Coding Tasks
- Quine: Realizing LLM Agents as Native POSIX Processes
- From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws
- SciDER: Scientific Data-centric End-to-end Researcher
- HarmonyCell: Automating Single-Cell Perturbation Modeling under Semantic and Distribution Shifts
- RepoRepair: Leveraging Code Documentation for Repository-Level Automated Program Repair
- VeRO: A Harness for Agents to Optimize Agents
- Can Generalist Agents Automate Data Curation?
- Hybrid-Gym: Training Coding Agents to Generalize Across Tasks
- Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems
- Beyond Isolated Tasks: A Framework for Evaluating Coding Agents on Sequential Software Evolution
- Fuzzing with Agents? Generators Are All You Need
- Composer 2 Technical Report
- Agent libOS: A Runtime Substrate for Capability-Controlled Self-Evolving LLM Agents
- AgentModernize: Preserving Business Logic in Legacy Modernization with Multi-Agent LLMs and Behavioral Specification Graphs
- Idleness is Relative: Exploiting Tool-Call Idle Windows for Offloading in Agentic Systems with MORI
- Sandlock: Confining AI Agent Code with Unprivileged Linux Primitives
- MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection
- SecRepoBench: Benchmarking Code Agents for Secure Code Completion in Real-World Repositories
- APWA: A Distributed Architecture for Parallelizable Agentic Workflows
- On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length
- Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable
- OpenThoughts-Agent: Data Recipes for Agentic Models
- Toward a Science of Intent: Closure Gaps and Delegation Envelopes for Open-World AI Agents
- PaperClaw: Harnessing Agents for Autonomous Research and Human-in-the-Loop Refinement
- Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle
- Can Coding Agents Reproduce Findings in Computational Materials Science?
- Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows
- The Buy-or-Build Decision, Revisited: How Agentic AI Changes the Economics of Enterprise Software
- GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)
- Memory Transfer Learning: How Memories are Transferred Across Domains in Coding Agents
- Spatial Atlas: Compute-Grounded Reasoning for Spatial-Aware Research Agent Benchmarks
- Spec Kit Agents: Context-Grounded Agentic Workflows
- ARIS: Autonomous Research via Adversarial Multi-Agent Collaboration
- AWCP: A Workspace Delegation Protocol for Deep-Engagement Collaboration across Remote Agents
- EvoMaster: A Foundational Evolving Agent Framework for Agentic Science at Scale
- Don't Let AI Agents YOLO Your Files: Shifting Information and Control to Filesystems for Agent Safety and Autonomy
- AgentSPEX: An Agent SPecification and EXecution Language
- MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems
- SWE-AGILE: A Software Agent Framework for Efficiently Managing Dynamic Reasoning Context
- Designing Digital Humans with Ambient Intelligence
- AgentSZZ: Teaching the LLM Agent to Play Detective with Bug-Inducing Commits
- AEGIS: White-Box Attack Path Generation using LLMs and Training Effectiveness Evaluation for Large-Scale Cyber Defence Exercises
- SERA: Soft-Verified Efficient Repository Agents
- Endless Terminals: Scaling RL Environments for Terminal Agents
- Self-Evolving Coding Agents
- LACE: Large Language Model Aided Multi-Agent Framework for Agile RISC-V Instruction Extension
- COMPAS: Difficulty-Aware Joint Search for Optimizing Code Generation
- Active-SWE: Benchmarking Coding Agents for Proactive Bug Fixing without Issue Reports
- Scrouting: Cost-Aware Routing of Coding Agents by Scouting the Repository First
- ReCodeAgent: A Multi-Agent Workflow for Language-agnostic Translation and Validation of Large-scale Repositories
- Frontier AI's Impact on the Cybersecurity Landscape
- Building LLM Agents by Incorporating Insights from Computer Systems
- Efficient Pretraining Length Scaling
- CRUST-Bench: A Comprehensive Benchmark for C-to-safe-Rust Transpilation
- SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugs
- Activated LoRA: Fine-tuned LLMs for Intrinsics
- DCAS: Decoupling CLI Agent Scaffolding to Internalize Planning across Scaffolds
- RepoOMP: Repository-Aware Hotspot OpenMP Parallelization via Dependency-Aware Context Reduction
- CodeGrep: An RL-Trained Retrieval Agent for LLM Coding Agents
- EcoAgent-Bench: Evaluating Economic Decision-Making in Budget-Constrained LLM Agents
- ASGE-RR: Agentic Service Graph Embedding with Revisable Reservations for Dynamic AI-Agent Calls
- Learning Globally Reusable Skills for Coding Agents
- Recursive Synthesis for Long-Horizon Terminal Tasks
- Matching Matters: A Fair Quality-Efficiency Benchmark for Command-Line Agents
- Scaffold-Mediated Post-Training: Co-Evolving Model Parameters and Procedural Scaffold Graphs
- Two Heads are Better Than One: Test-time Scaling of Multi-agent Collaborative Reasoning
- SeaView: Software Engineering Agent Visual Interface for Enhanced Workflow
- FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks
Discussions
Related