Towards a Science of AI Agent Reliability
2026/02/18 by Stephan Rabanser, Sayash Kapoor, Peter Kirgis +3 · 6 voices · 2 citations
#cs.AI #cs.CY #cs.LG
paper · pdf
Abstract
AI agents are increasingly deployed to execute important tasks. While rising accuracy scores on standard benchmarks suggest rapid progress, many agents still continue to fail in practice. This discrepancy highlights a fundamental limitation of current evaluations: compressing agent behavior into a single success metric obscures critical operational flaws. Notably, it ignores whether agents behave consistently across runs, withstand perturbations, fail predictably, or have bounded error severity. Grounded in safety-critical engineering, we provide a holistic performance profile by proposing twelve concrete metrics that decompose agent reliability along four key dimensions: consistency, robustness, predictability, and safety. Evaluating 15 models across two complementary benchmarks, we find that recent capability gains have only yielded small improvements in reliability. By exposing these persistent limitations, our metrics complement traditional evaluations while offering tools for reasoning about how agents perform, degrade, and fail.
Cited by
Discussions
- Very cool, thx! Had a chat with Fable about best practices in matching model to task, and it brought up this nifty paper I hadn’t seen from a Princeton group on reliability as a critically important a [bsky, 4 points, 1 comments]
- Towards a Science of AI Agent Reliability arxiv.org/abs/2602.16666 [bsky, 4 points, 0 comments]
- Interesting work that shows that frontier models are basically not improving in reliability despite them appearing to do better on benchmarks. Reliability is key to trust. Hard to imagine LLMs being t [bsky, 2 points, 0 comments]
- Towards a Science of AI Agent Reliability [hn, 2 points, 0 comments]
- Towards a Science of AI Agent Reliability arxiv.org/abs/2602.16666 - 연구자들은 정확도를 넘어 AI 에이전트의 신뢰성을 평가하는 프레임워크를 제안 - 안전 필수 공학에 기반하여 일관성, 견고성, 예측 가능성, 안전성의 4가지 차원에 걸쳐 12가지 지표를 도입 - 14개 모델을 두 벤치마크에서 테스트한 결 [bsky, 1 points, 0 comments]
- Towards a Science of AI Agent Reliability arxiv.org/abs/2602.16666 - 연구자들은 정확도를 넘어 AI 에이전트의 신뢰성을 평가하는 프레임워크를 제안 - 안전 필수 공학에 기반하여 일관성, 견고성, 예측 가능성, 안전성의 4가지 차원에 걸쳐 12가지 지표를 도입 - 14개 모델을 두 벤치마크에서 테스트한 결 [bsky, 1 points, 0 comments]
Related