WebGPT: Browser-assisted question-answering with human feedback
2021/12/17 by Reiichiro Nakano, Nakano, Reiichiro, Jacob Hilton +35 · 3 voices · 371 citations
Computer Science · Psychology · #Artificial intelligence #Computer science #Human–computer interaction #Imitation #Information retrieval #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Psychology #Quality (philosophy) #Task (project management) #Topic Modeling #World Wide Web #cs.AI #cs.CL #cs.LG
paper · pdf · doi:10.48550/arxiv.2112.09332
published in arXiv (Cornell University) (Cornell University) · 32 pages
openalex publication_date 2021/12/17 · openalex created_date 2022/05/05 · arxiv created 2022/06/01 · arxiv updated 2022/06/03 · openalex updated_date 2026/08/05
Abstract
We fine-tune GPT-3 to answer long-form questions using a text-based web-browsing environment, which allows the model to search and navigate the web. By setting up the task so that it can be performed by humans, we are able to train models on the task using imitation learning, and then optimize answer quality with human feedback. To make human evaluation of factual accuracy easier, models must collect references while browsing in support of their answers. We train and evaluate our models on ELI5, a dataset of questions asked by Reddit users. Our best model is obtained by fine-tuning GPT-3 using behavior cloning, and then performing rejection sampling against a reward model trained to predict human preferences. This model's answers are preferred by humans 56% of the time to those of our human demonstrators, and 69% of the time to the highest-voted answer from Reddit.
Cited by
- WebCoach: Self-Evolving Web Agents with Cross-Session Memory Guidance
- Answer-Reconstruction Search Density: Measuring the Query and Source Work Compressed by Conversational Answers
- Do Data Agents Need Semantic Metadata? A Comparative Study in Agentic Data Retrieval
- ClawBench: Can AI Agents Complete Everyday Online Tasks?
- DeLIVeR: Decomposed Learning for Information-grounded Veracity Recognition via Reinforced Knowledge Graph Exploration
- The Behavioral Credibility Trilemma: When Calibrated Autonomy Becomes Impossible
- DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments
- Do Generative AI Assistants Respect robots.txt? Tracing Web Access Beyond Visible Answers
- RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources
- AnovaX: A Local, Multi-Agent Voice Assistant with LLM Planning, Typed Executors, and Adaptive Recovery
- Multi-Turn On-Policy Distillation with Prefix Replay
- SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaboration
- FormulaSPIN: Self-Play Fine-Tuning for Natural Language to Spreadsheet Formula Generation
- Hallucinations Undermine Trust; Metacognition is a Way Forward
- Skill-Contracted Agents for Evidence-Aware Materials Literature Analysis
- Calibrated Selective Fact-Checking via Evidence Chain Evaluation
- Learning to Orchestrate Agents in Natural Language with the Conductor
- Layer-0 Suppressors Ground Hallucination Inevitability: A Mechanistic Account of How Transformers Trade Factuality for Hedging
- K2-Think: A Parameter-Efficient Reasoning System
- Coral Protocol: Open Infrastructure Connecting The Internet of Agents
- It's LIT! Reliability-Optimized LLMs with Inspectable Tools
- SPIRAL: Symbolic LLM Planning via Grounded and Reflective Search
- Video-BrowseComp: Benchmarking Agentic Video Research on Open Web
- Agent2World: Learning to Generate Symbolic World Models via Adaptive Multi-Agent Feedback
- Unbiased Visual Reasoning with Controlled Visual Inputs
- ConsistencyGate: Preventing Memory Contamination in LLM Agents via Self-Consistency Admission Control
- Inverse RL Helps Align AI by Imitating Humans
- From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World
- The Cartesian Cut in Agentic AI
- An Information Theoretic Perspective on Agentic System Design
- A Unified Definition of Hallucination: It's The World Model, Stupid!
- AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agent
- AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning
- Stackelberg Learning from Human Feedback: Preference Optimization as a Sequential Game
- PDE-Agent: A toolchain-augmented multi-agent framework for PDE solving
- Let's (not) just put things in Context: Test-Time Training for Long-Context LLMs
- AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning
- VERAFI: Verified Agentic Financial Intelligence through Neurosymbolic Policy Generation
- Source Coverage and Citation Bias in LLM-based vs. Traditional Search Engines
- ValuePilot: A Two-Phase Framework for Value-Driven Decision-Making
- DART: Leveraging Multi-Agent Disagreement for Tool Recruitment in Multimodal Reasoning
- On the Limits of Test-Time Compute: Sequential Reward Filtering for Better Inference
- Overcoming State Inertia: Minimally Invasive Temporal Alignment for Evolving Contexts
- Process-Centric Analysis of Agentic Software Systems
- Evolving Paradigms in Task-Based Search and Learning: A Comparative Analysis of Traditional Search Engine with LLM-Enhanced Conversational Search System
- An Empirical Study on the Security Vulnerabilities of GPTs
- ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
- VeriSciQA: An Auto-Verified Dataset for Scientific Visual Question Answering
- Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalization
- CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization
- SPINE: Token-Selective Test-Time Reinforcement Learning with Entropy-Band Regularization
- Goal-Directed Search Outperforms Goal-Agnostic Memory Compression in Long-Context Memory Tasks
- Fast LLM Post-training via Decoupled and Fastest-of-N Speculation
- Finetuning LLMs for Automatic Form Interaction on Web-Browser in Selenium Testing Framework
- Mobile-Agent-RAG: Driving Smart Multi-Agent Coordination with Contextual Knowledge Empowerment for Long-Horizon Mobile Automation
- Convergence and Stability Analysis of Self-Consuming Generative Models with Heterogeneous Human Curation
- AlphaCast: A Human Wisdom-LLM Intelligence Co-Reasoning Framework for Interactive Time Series Forecasting
- OSGym: Scalable OS Infra for Computer Use Agents
- From Experience to Strategy: Empowering LLM Agents with Trainable Graph Memory
- IterResearch: Rethinking Long-Horizon Agents via Markovian State Reconstruction
- Reinforcement Learning Improves Traversal of Parametric Knowledge in LLMs
- Inference-Time Personalized Alignment with a Few User Preference Queries
- Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch
- Towards Understanding, Analyzing, and Optimizing Agentic AI Execution: A CPU-Centric Perspective
- GraphChain: Large Language Models for Large-scale Graph Analysis via Tool Chaining
- PORTool: Tool-Use LLM Training with Rewarded Tree
- Decomposition-Enhanced Training for Post-Hoc Attributions In Language Models
- Truth-Aware Decoding: A Program-Logic Approach to Factual Language Generation
- What Does It Take to Detect an AI Agent? Minimal Feature Sets for Behavioral Detection under Browser Automation
- GroundEval: A Deterministic Replacement for LLM-as-Judge in Stateful Agent Evaluation
- Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agents
- A Survey of Data Agents: Emerging Paradigm or Overstated Hype?
- The Best of N Worlds: Aligning Reinforcement Learning with Best-of-N Sampling via max@k Optimisation
- Adaptive Blockwise Search: Inference-Time Alignment for Large Language Models
- Multi-Stakeholder Alignment in LLM-Powered Collaborative AI Systems: A Multi-Agent Framework for Intelligent Tutoring
- Beyond Reasoning Gains: Mitigating General Capabilities Forgetting in Large Reasoning Models
- Reducing the Probability of Undesirable Outputs in Language Models Using Probabilistic Inference
- PanicToCalm: A Proactive Counseling Agent for Panic Attacks
- Re-evaluating Minimum Bayes Risk Decoding for Automatic Speech Recognition
- Surfer 2: The Next Generation of Cross-Platform Computer Use Agents
- CUARewardBench: A Benchmark for Evaluating Reward Models on Computer-using Agent
- Crucible: Quantifying the Potential of Control Algorithms through LLM Agents
- Contextual Attention Modulation: Towards Efficient Multi-Task Adaptation in Large Language Models
- Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents
- Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing
- WEBSERV: A Full-Stack and RL-Ready Web Environment for Training Web Agents at Scale
- Natural Language Tools: A Natural Language Approach to Tool Calling In Large Language Agents
- NetMCP: Network-Aware Model Context Protocol Platform for LLM Capability Extension
- Grounding Long-Context Reasoning with Contextual Normalization for Retrieval-Augmented Generation
- Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism
- Information-Theoretic Reward Modeling for Stable RLHF: Detecting and Mitigating Reward Hacking
- On the Role of Preference Variance in Preference Optimization
- How Well Can Preference Optimization Generalize Under Noisy Feedback?
- Attacks by Content: Automated Fact-checking is an AI Security Issue
- SusBench: An Online Benchmark for Evaluating Dark Pattern Susceptibility of Computer-Use Agents
- A Survey on Agentic Multimodal Large Language Models
- PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literature
- LLM×MapReduce-V3: Enabling Interactive In-Depth Survey Generation through a MCP-Driven Hierarchically Modular Agent System
- Known By Their Actions: Fingerprinting LLM Browser Agents via UI Traces
- Can RL Improve Generalization of LLM Agents? An Empirical Study
- Safety Game: Balancing Safe and Informative Conversations with Blackbox Agentic AI using LP Solvers
- Fundamentals of Building Autonomous LLM Agents
- MATRIX: Multimodal Agent Tuning for Robust Tool-Use Reasoning
- FlowSearch: Advancing deep research with dynamic structured knowledge flow
- Memory Retrieval and Consolidation in Large Language Models through Function Tokens
- CREST-Search: Comprehensive Red-teaming for Evaluating Safety Threats in Large Language Models Powered by Web Search
- xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learning
- CommandSans: Securing AI Agents with Surgical Precision Prompt Sanitization
- Prepared mind, fast response: A temporal decoupling framework for adaptive knowledge orchestration in open-domain dialogue
- Red-Bandit: Test-Time Adaptation for LLM Red-Teaming via Bandit-Guided LoRA Experts
- Tool-Augmented Policy Optimization: Synergizing Reasoning and Adaptive Tool Use with Reinforcement Learning
- MIRA: Towards Mitigating Reward Hacking in Inference-Time Alignment of T2I Diffusion Models
- Efficient Tree-Structured Deep Research with Adaptive Resource Allocation
- Exposing Citation Vulnerabilities in Generative Engines
- InfoMosaic-Bench: Evaluating Multi-Source Information Seeking in Tool-Augmented Agents
- Optimal Stopping vs Best-of-N for Inference Time Optimization
- Test-Time Scaling in Diffusion LLMs via Hidden Semi-Autoregressive Experts
- MARS: Co-evolving Dual-System Deep Research via Multi-Agent Reinforcement Learning
- AlphaApollo: A System for Deep Agentic Reasoning
- Best of mini-N in-loop Sampling: A Contextual Quality Reward Model for Reliable and Efficient Best-of-N Sampling
- JEF-Hinter: Leveraging Offline Knowledge for Improving Web Agents Adaptation
- Best-of-Majority: Minimax-Optimal Strategy for Pass@k Inference Scaling
- AgenticRAG: Tool-Augmented Foundation Models for Zero-Shot Explainable Recommender Systems
- Demystifying deep search: a holistic evaluation with hint-free multi-hop questions and factorised metrics
- PAL-UI: Planning with Active Look-back for Vision-Based GUI Agents
- Limited Preference Data? Learning Better Reward Model with Latent Space Synthesis
- A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experiments
- Planner-R1: Reward Shaping Enables Efficient Agentic RL with Smaller LLMs
- Detecting and Fixing API Misuses of Data Science Libraries Using Large Language Models
- Structural Reward Model: Enhancing Interpretability, Efficiency, and Scalability in Reward Modeling
- Humanline: Online Alignment as Perceptual Loss
- Not Wrong, But Untrue: LLM Overconfidence in Document-Based Queries
- Mix-Ecom: Towards Mixed-Type E-Commerce Dialogues with Complex Domain Rules
- SafeSearch: Automated Red-Teaming for the Safety of LLM-Based Search Agents
- Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment
- Large-Scale Constraint Generation -- Can LLMs Parse Hundreds of Constraints?
- PARL-MT: Learning to Call Functions in Multi-Turn Conversation with Progress Awareness
- Dialogues with AI Reduce Beliefs in Misinformation but Build No Lasting Discernment Skills
- Fine-Grained Detection of Context-Grounded Hallucinations Using LLMs
- What Should I Cite? A RAG Benchmark for Academic Citation Prediction
- It's Not You, It's Clipping: A Soft Trust-Region via Probability Smoothing for LLM RL
- Hallucination-Resistant, Domain-Specific Research Assistant with Self-Evaluation and Vector-Grounded Retrieval
- ToolBrain: A Flexible Reinforcement Learning Framework for Agentic Tools
- SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them
- DeepResearch Agent System
- Harness-G: A Graph-Structured Harness for Search Agents
- Dr-DCI: Scaling Direct Corpus Interaction via Dynamic Workspace Expansion
- Benchmarking Open-Ended Multi-Agent Coordination in Language Agents
- Reflect before Act: Proactive Error Correction in Language Models
- CoCoA: Confidence and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in Large Language Models
- UIPro: Unleashing Superior Interaction Capability For GUI Agents
- Towards General Computer Control with Hierarchical Agents and Multi-Level Action Spaces
- Asking a Language Model for Diverse Responses
- SignalLLM: A General-Purpose LLM Agent Framework for Automated Signal Processing
- Governing Automated Strategic Intelligence
- SEFRQO: A Self-Evolving Fine-Tuned RAG-Based Query Optimizer
- RLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformation
- A Framework for Generating Artificial Datasets to Validate Absolute and Relative Position Concepts
- SIRAG: Towards Stable and Interpretable RAG with A Process-Supervised Multi-Agent Framework
- Environmental Injection Attacks against GUI Agents in Realistic Dynamic Environments
- DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL
- WebSight: A Vision-First Architecture for Robust Web Agents
- Decoding Alignment: A Critical Survey of LLM Development Initiatives through Value-setting and Data-centric Lens
- Comp-X: On Defining an Interactive Learned Image Compression Paradigm With Expert-driven LLM Agent
- VehicleWorld: A Highly Integrated Multi-Device Environment for Intelligent Vehicle Interaction
- Icon2: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulation
- Towards a Unified View of Large Language Model Post-Training
- Explainable Knowledge Graph Retrieval-Augmented Generation (KG-RAG) with KG-SMILE
- DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence
- Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
- EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notes
- Can Compact Language Models Search Like Agents? Distillation-Guided Policy Optimization for Preserving Agentic RAG Capabilities
- ReSURE: Regularizing Supervision Unreliability for Multi-turn Dialogue Fine-tuning
- Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs
- From Bits to Boardrooms: A Cutting-Edge Multi-Agent LLM Framework for Business Excellence
- Foundational Design Principles and Patterns for Building Robust and Adaptive GenAI-Native Systems
- Better Language Model-Based Judging Reward Modeling through Scaling Comprehension Boundaries
- Select to Know: An Internal-External Knowledge Self-Selection Framework for Domain-Specific Question Answering
- Multimodal Data Storage and Retrieval for Embodied AI: A Survey
- A Functionality-Grounded Benchmark for Evaluating Web Agents in E-commerce Domains
- Deep Research: A Survey of Autonomous Research Agents
- Fast, Slow, and Tool-augmented Thinking for LLMs: A Review
- DistFlow: A Fully Distributed RL Framework for Scalable and Efficient LLM Post-Training
- Thinking Inside the Mask: In-Place Prompting in Diffusion LLMs
- MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents
- Improving and Evaluating Open Deep Research Agents
- OpenCUA: Open Foundations for Computer-Use Agents
- Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression
- HGMF: A Hierarchical Gaussian Mixture Framework for Scalable Tool Invocation within the Model Context Protocol
- Remote Sensing Image Intelligent Interpretation with the Language-Centered Perspective: Principles, Methods and Challenges
- The NordDRG AI Benchmark for Large Language Models
- Exploring a biocentric LLM-based assistant in environmental decision-making with more-than-human representation of the Tagus Estuary
- Can Large Models Fool the Eye? A New Turing Test for Biological Animation
- SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience
- Multi-module GRPO: Composing Policy Gradients and Prompt Optimization for Language Model Programs
- Chain of Questions: Guiding Multimodal Curiosity in Language Models
- Large Language Model's Multi-Capability Alignment in Biomedical Domain
- ToolGrad: Efficient Tool-use Dataset Generation with Textual "Gradients"
- AttnTrace: Attention-based Context Traceback for Long-Context LLMs
- SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents
- CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions
- MetaAgent: Toward Self-Evolving Agent via Tool Meta-Learning
- BAR Conjecture: the Feasibility of Inference Budget-Constrained LLM Services with Authenticity and Reasoning
- Phi-Ground Tech Report: Advancing Perception in GUI Grounding
- Improving Generative Ad Text on Facebook using Reinforcement Learning
- Understanding Human Limits in Pattern Recognition: A Computational Model of Sequential Reasoning in Rock, Paper, Scissors
- Trustworthy Reasoning: Evaluating and Enhancing Factual Accuracy in LLM Intermediate Thought Processes
- A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions
- Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
- Theoretical Foundations and Mitigation of Hallucination in Large Language Models
- PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation
- Thinking Isn't an Illusion: Overcoming the Limitations of Reasoning Models via Tool Augmentations
- Deliberative Searcher: Improving LLM Reliability via Reinforcement Learning with constraints
- Granular feedback merits sophisticated aggregation
- Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs
- SAND: Boosting LLM Agents with Self-Taught Action Deliberation
- Bradley-Terry and Multi-Objective Reward Modeling Are Complementary
- Multi-Agent Retrieval-Augmented Framework for Evidence-Based Counterspeech Against Health Misinformation
- Integrating External Tools with Large Language Models to Improve Accuracy
- Agentic-R1: Distilled Dual-Strategy Reasoning
- Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents
- WebArXiv: Evaluating Multimodal Agents on Time-Invariant arXiv Tasks
- TeamCMU at Touché: Adversarial Co-Evolution for Advertisement Integration and Detection in Conversational Search
- iPanda: An LLM-based Agent for Automated Conformance Testing of Communication Protocols
- Positional Bias in Binary Question Answering: How Uncertainty Shapes Model Preferences
- Agent.xpu: Efficient Scheduling of Agentic LLM Workloads on Heterogeneous SoC
- BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute
- KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model
- MMSearch-R1: Incentivizing LMMs to Search
- Context Attribution with Multi-Armed Bandit Optimization
- Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track
- Inference-Time Reward Hacking in Large Language Models
- From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents
- Deep Research Agents: A Systematic Examination And Roadmap
- Aligning Frozen LLMs by Reinforcement Learning: An Iterative Reweight-then-Optimize Approach
- Beyond Syntax: Action Semantics Learning for App Agents
- Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models
- Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examples
- Next-Token Prediction Should be Ambiguity-Sensitive: A Meta-Learning Perspective
- Reranking-based Generation for Unbiased Perspective Summarization
- MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents
- Modeling the One-to-Many Property in Open-Domain Dialogue with LLMs
- OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents
- SimSpark: Interactive Simulation of Social Media Behaviors
- LexiMark: Robust Watermarking via Lexical Substitutions to Enhance Membership Verification of an LLM's Textual Training Data
- Fake it till You Make it: Reward Modeling as Discriminative Prediction
- Human-assisted Robotic Policy Refinement via Action Preference Optimization
- Leveraging In-Context Learning for Language Model Agents
- Min-p, Max Exaggeration: A Critical Analysis of Min-p Sampling in Language Models
- Adaptive Batch-Wise Sample Scheduling for Direct Preference Optimization
- C-SEO Bench: Does Conversational SEO Work?
- AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Search
- Micro-Act: Mitigating Knowledge Conflict in LLM-based RAG via Actionable Self-Reasoning
- Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcement
- From Standalone LLMs to Integrated Intelligence: A Survey of Compound Al Systems
- Truly Self-Improving Agents Require Intrinsic Metacognitive Learning
- Kinetics: Rethinking Test-Time Scaling Laws
- AI Agents for Conversational Patient Triage: Preliminary Simulation-Based Evaluation with Real-World EHR Data
- Advancing and Benchmarking Personalized Tool Invocation for LLMs
- Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Models
- TracLLM: A Generic Framework for Attributing Long Context LLMs
- Automated Web Application Testing: End-to-End Test Case Generation with Large Language Models and Screen Transition Graphs
- Surfer-H Meets Holo1: Cost-Efficient Web Agent Powered by Open Weights
- Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports From Scratch with Agentic Framework
- AI-Native Brand Identity: From Visual Recognition to Cryptographic Verification
- Cycle Consistency as Reward: Learning Image-Text Alignment without Human Preferences
- Assessing and Enhancing the Robustness of LLM-based Multi-Agent Systems Through Chaos Engineering
- Psi-Sampler: Initial Particle Sampling for SMC-Based Inference-Time Reward Alignment in Score Models
- RewardBench 2: Advancing Reward Model Evaluation
- MCP-Zero: Active Tool Discovery for Autonomous LLM Agents
- HADA: Human-AI Agent Decision Alignment Architecture
- Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing
- Soft Best-of-n Sampling for Model Alignment
- Open CaptchaWorld: A Comprehensive Web-based Platform for Testing and Benchmarking Multimodal LLM Agents
- RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generation
- When Large Multimodal Models Confront Evolving Knowledge: Challenges and Explorations
- Proximalized Preference Optimization for Diverse Feedback Types: A Decomposed Perspective on DPO
- Bounded Rationality for LLMs: Satisficing Alignment at Inference-Time
- AgentAlign: Navigating Safety Alignment in the Shift from Informative to Agentic Large Language Models
- Text2Grad: Reinforcement Learning from Natural Language Feedback
- RISE: Reasoning Enhancement via Iterative Self-Exploration in Multi-hop Question Answering
- On Learning Verifiers and Implications to Chain-of-Thought Reasoning
- Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generation
- CogniBench: A Legal-inspired Framework and Dataset for Assessing Cognitive Faithfulness of Large Language Models
- CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic
- Token-level Accept or Reject: A Micro Alignment Approach for Large Language Models
- Select, Read, and Write: A Multi-Agent Framework of Full-Text-based Related Work Generation
- Inference-time Alignment in Continuous Space
- Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewards
- Understanding the Performance Gap in Preference Learning: A Dichotomy of RLHF and DPO
- SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safety
- A Survey on Progress in LLM Alignment from the Perspective of Reward Design
- ChartLens: Fine-grained Visual Attribution in Charts
- ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignment
- Retrieval-Augmented Generation for Service Discovery: Chunking Strategies and Benchmarking
- SearchMaster: Grounded and Regulated Self-Play for Search Agents
- Scalable Valuation of Human Feedback through Provably Robust Model Alignment
- RaDeR: Reasoning-aware Dense Retrieval Models
- Dynamic Risk Assessments for Offensive Cybersecurity Agents
- Fetch-then-Explore: Decoupling Selection from Extraction over a Persistent Workspace for Search Agents
- VerifyBench: Benchmarking Reference-based Reward Systems for Large Language Models
- Before Reasoning Can Fail: Pre-Evidence Procedural Failures in Agentic RAG
- Structured Agent Distillation for Large Language Model
- Transparent and Robust RAG: Adaptive-Reward Reinforcement Learning for Decision Traceability
- Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis
- The Hidden Dangers of Browsing AI Agents
- Computing with Agentic Oracles
- Web Intellectual Property at Risk: Preventing Unauthorized Real-Time Retrieval by Large Language Models
- Rethinking Stateful Tool Use in Multi-Turn Dialogues: Benchmarks and Challenges
- Pairwise Calibrated Rewards for Pluralistic Alignment
- AdaBoN: Adaptive Best-of-N Alignment
- LLM Agents Are Hypersensitive to Nudges
- Deep Research Pretraining via Predictive Navigation
- Multi-Dimensional Assessment for AI Cognition (MAAC): A Theoretical Framework for Process-Oriented Cognitive Evaluation of Text-Based AI Systems
- PeerGuard: Defending Multi-Agent Systems Against Backdoor Attacks Through Mutual Reasoning
- Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correction
- Demystifying AI Agents: The Final Generation of Intelligence
- RMSWeb: Reflection, Failure-Mode Mining, and Salvage-DS for Web Agent Reinforcement Learning
- Scent of Knowledge: Optimizing Search-Enhanced Reasoning with Information Foraging
- Ethics and Persuasion in Reinforcement Learning from Human Feedback: A Procedural Rhetorical Approach
- HealthBench: Evaluating Large Language Models Towards Improved Human Health
- FitText: Evolving Agent Tool Ecologies via Memetic Retrieval
- Large Language Models for Computer-Aided Design: A Survey
- ToolACE-DEV: Self-Improving Tool Learning via Decomposition and EVolution
- Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving
- Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents
- AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents
- Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes
- Tail-Aware Information-Theoretic Bounds for LLM Alignment under Heavy-Tailed Rewards
- Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents
- Visual Test-time Scaling for GUI Agent Grounding
- The Anatomy of the Moltbook Social Graph
- Decoupling Search from Reasoning: A Vendor-Agnostic Grounding Architecture for LLM Agents
- Generative Engine Optimization: A VLM and Agent Framework for Pinterest Acquisition Growth
- AgentOS: From Application Silos to a Natural Language-Driven Data Ecosystem
- The Synthetic Web: Adversarially-Curated Mini-Internets for Diagnosing Epistemic Weaknesses of Language Agents
- Agentic Test-Time Scaling for WebAgents
- PIIGuard: Mitigating PII Harvesting under Adversarial Sanitization
- From Citation Selection to Citation Absorption: A Measurement Framework for Generative Engine Optimization Across AI Search Platforms
- PaperClaw: Harnessing Agents for Autonomous Research and Human-in-the-Loop Refinement
- OpenSkill: Open-World Self-Evolution for LLM Agents
- Can Virtual Agents Care? Designing an Empathetic and Personalized LLM-Driven Conversational Agent
- Towards Recursive Self-Evolving Agentic Literature Retrieval
- MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wild
- Reversible Lifelong Model Editing via Semantic Routing-Based LoRA
- Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning
- Faithfulness vs. Safety: Evaluating LLM Behavior Under Counterfactual Medical Evidence
- BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese
- AndroidGen: Building an Android Language Agent under Data Scarcity
- Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning
- InfoSeeker: A Scalable Hierarchical Parallel Agent Framework for Web Information Seeking
- Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests
- DataParasite Enables Scalable and Repurposable Online Data Curation
- A Comprehensive Survey of Knowledge-Based Vision Question Answering Systems: The Lifecycle of Knowledge in Visual Reasoning Task
- Search, Inspect, Fetch: Exploiting Structure-Aware Boolean Retrieval for Deep-Research Agents
- FinReportBench: Measuring and Improving Institution-Grade Financial Report Generation
- Blockchain Empowered Trustworthy Agent Networks: Foundations, Taxonomy, and Future Directions
- TraveLLaMA: A Multimodal Travel Assistant with Large-Scale Dataset and Structured Reasoning
- Building LLM Agents by Incorporating Insights from Computer Systems
- TTRL: Test-Time Reinforcement Learning
- Establishing Reliability Metrics for Reward Models in Large Language Models
- a1: Steep Test-time Scaling Law via Environment Augmented Generation
- Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo
- Memorization vs. Reasoning: Updating LLMs with New Knowledge
- BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents
- SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents
- Offline Learning and Forgetting for Reasoning with Large Language Models
- DeepTrans: Deep Reasoning Translation via Reinforcement Learning
- Learning from Reference Answers: Versatile Language Model Alignment without Binary Human Preference Data
- UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents
- AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories
- Defense against Prompt Injection Attacks via Mixture of Encodings
- Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling
Discussions
Related