What's the next frontier for Data-centric AI? Data Savvy Agents

2025/11/02 by Nabeel Seedat, Seedat, Nabeel, Jiashuo Liu +3
Computer Science · Decision Sciences · Social Sciences · #Data Quality and Management #Ethics and Social Impacts of AI #FOS: Computer and information sciences #Machine Learning (cs.LG) #Mobile Crowdsensing and Crowdsourcing

paper · pdf · doi:10.48550/arxiv.2511.01015

openalex publication_date 2025/11/02 · openalex created_date 2025/11/06 · openalex updated_date 2026/07/28

Abstract

The recent surge in AI agents that autonomously communicate, collaborate with humans and use diverse tools has unlocked promising opportunities in various real-world settings. However, a vital aspect remains underexplored: how agents handle data. Scalable autonomy demands agents that continuously acquire, process, and evolve their data. In this paper, we argue that data-savvy capabilities should be a top priority in the design of agentic systems to ensure reliable real-world deployment. Specifically, we propose four key capabilities to realize this vision: (1) Proactive data acquisition: enabling agents to autonomously gather task-critical knowledge or solicit human input to address data gaps; (2) Sophisticated data processing: requiring context-aware and flexible handling of diverse data challenges and inputs; (3) Interactive test data synthesis: shifting from static benchmarks to dynamically generated interactive test data for agent evaluation; and (4) Continual adaptation: empowering agents to iteratively refine their data and background knowledge to adapt to shifting environments. While current agent research predominantly emphasizes reasoning, we hope to inspire a reflection on the role of data-savvy agents as the next frontier in data-centric AI.

Citations

• Towards Human-Guided, Data-Centric LLM Co-Pilots
• YETI (YET to Intervene) Proactive Interventions by Multimodal AI Agents in Augmented Reality Tasks
• Lifelong Learning of Large Language Model based Agents: A Roadmap
• TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks
• Adapting to Non-Stationary Environments: Multi-Armed Bandit Enhanced Retrieval-Augmented Generation on Knowledge Graphs
• Human-In-the-Loop Software Development Agents
• Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks
• Context-Aware Testing: A New Paradigm for Model Testing with Large Language Models
• Matchmaker: Self-Improving Large Language Model Programs for Schema Matching
• Advancing Agentic Systems: Dynamic Task Decomposition, Tool Integration and Evaluation using Novel Metrics and Dataset
• Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance
• CliMB: An AI-enabled Partner for Clinical Predictive Modeling
• Scaling Retrieval-Based Language Models with a Trillion-Token Datastore
• You can't handle the (dirty) truth: Data-centric insights improve pseudo-labeling
• Revisiting Catastrophic Forgetting in Large Language Model Tuning
• CleanAgent: Automating Data Standardization with LLM-based Agents
• Dissecting Sample Hardness: A Fine-Grained Analysis of Hardness Characterization Methods for Data-Centric AI
• Data Interpreter: An LLM Agent For Data Science
• DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning
• A hierarchical decomposition for explaining ML performance discrepancies
• Large Language Model-based Human-Agent Collaboration for Complex Task Solving
• A Survey on Context-Aware Multi-Agent Systems: Techniques, Challenges and Future Directions
• WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
• Retrieval-Augmented Generation for Large Language Models: A Survey
• Data Acquisition: A New Frontier in Data-centric AI
• TRIAGE: Characterizing and auditing training data for improved regression
• An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
• AgentBench: Evaluating LLMs as Agents
• Anticipatory Thinking Challenges in Open Worlds: Risk Management
• OpenDataVal: a Unified Benchmark for Data Valuation
• Generative Agents: Interactive Simulacra of Human Behavior
• Data-centric Artificial Intelligence: A Survey
• Diagnosing Model Performance Under Distribution Shift
• A Comprehensive Survey of Continual Learning: Theory, Method and Application
• Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data
• REPTILE: A Proactive Real-Time Deep Reinforcement Learning Self-adaptive Framework
• Domino: Discovering Systematic Errors with Cross-Modal Embeddings
• AI Autonomy : Self-Initiated Open-World Continual Learning and Adaptation
• Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
• A Data Quality-Driven View of MLOps
• Three scenarios for continual learning
• Learning Active Learning from Data
• VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
• DeepNote: Note-Centric Deep Retrieval-Augmented Generation
• LADDER: Language-Driven Slice Discovery and Error Rectification in Vision Classifiers

Related

• A Survey of Data Agents: Emerging Paradigm or Overstated Hype?
• Autonomous Data Agents: A New Opportunity for Smart Data
• Supporting Our AI Overlords: Redesigning Data Systems to be Agent-First
• Data Agent: A Holistic Architecture for Orchestrating Data+AI Ecosystems
• Data-centric AI: Perspectives and Challenges