2025/10/05 by Mayank Ravishankara, Ravishankara, Mayank, Varindra V. Persad Maharaj +1 · 1 voice · 1 citation
Computer Science · #Intelligent Tutoring Systems and Adaptive Learning
paper · doi:10.1109/access.2025.3649182
openalex created_date 2025/10/09 · openalex publication_date 2025/12/29 · openalex updated_date 2026/07/28
This survey paper chronicles the evolution of evaluation in multimodal artificial intelligence (AI), framing it as a progression of increasingly sophisticated "cognitive examinations." We argue that the field is undergoing a paradigm shift, moving from simple recognition tasks that test "what" a model sees, to complex reasoning benchmarks that probe "why" and "how" it understands. This evolution is driven by the saturation of older benchmarks, where high performance often masks fundamental weaknesses. We chart the journey from the foundational "knowledge tests" of the ImageNet era to the "applied logic and comprehension" exams such as GQA and Visual Commonsense Reasoning (VCR), which were designed specifically to diagnose systemic flaws such as shortcut learning and failures in compositional generalization. We then survey the current frontier of "expert-level integration" benchmarks (e.g., MMBench, SEED-Bench, MMMU) designed for today’s powerful multimodal large language models (MLLMs). Next, we explore the uncharted territories of evaluating abstract, creative, and social intelligence. We also discuss the emerging "Post-2025" landscape, where rapid saturation of reasoning benchmarks by models like Gemini 3 Pro necessitates a shift toward "living" adversarial evaluations. We conclude that the narrative of AI evaluation is not merely a history of datasets, but a continuous, adversarial process of designing better examinations that, in turn, redefine our goals for creating truly intelligent systems.