Effectiveness of Intelligent Tutoring Systems
2015/04/17 by James A. Kulik, J. D. Fletcher · 1 voice · 812 citations
Computer Science · Mathematics · Psychology · #Artificial intelligence #Computer science #Control (management) #Implementation #Innovative Teaching and Learning Methods #Intelligent Tutoring Systems and Adaptive Learning #Intelligent decision support system #Intelligent tutoring system #Machine learning #Mathematics #Online Learning and Analytics #Percentile #Software engineering #Statistics #Test (biology)
paper · doi:10.3102/0034654315581420
published in Review of Educational Research 86(1), 42-78 (SAGE Publishing)
openalex publication_date 2015/04/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Abstract
This review describes a meta-analysis of findings from 50 controlled evaluations of intelligent computer tutoring systems. The median effect of intelligent tutoring in the 50 evaluations was to raise test scores 0.66 standard deviations over conventional levels, or from the 50th to the 75th percentile. However, the amount of improvement found in an evaluation depended to a great extent on whether improvement was measured on locally developed or standardized tests, suggesting that alignment of test and instructional objectives is a critical determinant of evaluation results. The review also describes findings from two groups of evaluations that did not meet all of the selection requirements for the meta-analysis: six evaluations with nonconventional control groups and four with flawed implementations of intelligent tutoring systems. Intelligent tutoring effects in these evaluations were small, suggesting that evaluation results are also affected by the nature of control treatments and the adequacy of program implementations.
Citations
Cited by
- Beyond "Fairness:" Structural (In)justice Lenses on AI for Education
- Enhancing School Students’ Self-Regulated Learning through Generative AI Support: A Randomized Controlled Trial
- Computer-assisted learning in the real world: How Khan Academy influences student math learning
- (When) do teacher professional development interventions improve student Achievement? A meta-analysis of 128 high-quality studies
- Asking questions matters: comparing the effect of learning through dialogue with peers, ChatGPT, and intelligent tutoring system
- When Machines Join the Moral Circle: The Persona Effect of Generative AI Agents in Collaborative Reasoning
- Beyond Good AI: The Need for Sound Learning Theories in AIED
- ChatGPT in Education: An Effect in Search of a Cause
- Artificial intelligence in education: A systematic literature review
- A Systematic Review of Research on Personalized Learning: Personalized by Whom, to What, How, and for What Purpose(s)?
- Judgments of learning in the wild: Establishing ecological validity with an intelligent tutoring system in a field study
- Effects of artificial intelligence learning environments on engineers’ intercultural communication competence: mediating role of learning adaptability
- Emotional Artificial Intelligence in Education: A Systematic Review and Meta-Analysis
- The Rise of Thinking Machines: A Review of Artificial Intelligence in Contemporary Communication
- The interplay of learning, analytics and artificial intelligence in education: A vision for hybrid intelligence
- State of the art and practice in AI in education
- Experimentally Testing AI-Powered Content Transformations on Student Learning
- Feed-O-Meter: Investigating AI-Generated Mentee Personas as Interactive Agents for Scaffolding Design Feedback Practice
- MultiGen: Child-Friendly Multilingual Speech Generator with LLMs
- Adaptive teaching with technology enhances lasting learning
- AI Tutors vs. Tenacious Myths: Evidence from Personalised Dialogue Interventions in Education
- Augmented Reality, Virtual Reality, and Intelligent Tutoring Systems in Education and Training: A Systematic Literature Review
- The effectiveness of intelligent tutoring systems on K‐12 students' reading comprehension: A meta‐analysis
- Artificial Intelligence in Technology-Enhanced Assessment: A Survey of Machine Learning
- Technology-based interactive guidance to promote learning performance and self-regulation: a chatbot-assisted self-regulated learning approach
- An application of Chinese dialogue-based intelligent tutoring system in remedial instruction for mathematics learning
- Efficacy of a Computer Tutor that Models Expert Human Tutors
- Effectiveness of Adaptive E-Learning Environments on Knowledge, Competence, and Behavior in Health Professionals and Students: Protocol for a Systematic Review and Meta-Analysis. [europepmc]
- SKOPE-IT (Shareable Knowledge Objects as Portable Intelligent Tutors): overlaying natural language tutoring on an adaptive learning system for mathematics. [europepmc]
- Development and application of a multi-modal task analysis to support intelligent tutoring of complex skills. [europepmc]
- Comments and reflections on ITS and STEM education and training. [europepmc]
- Comparison and Efficacy of Synergistic Intelligent Tutoring Systems with Human Physiological Response. [europepmc]
- Online Diagnostic Assessment in Support of Personalized Teaching and Learning: The eDia System. [europepmc]
- Differentiated Instruction in Secondary Education: A Systematic Review of Research Evidence. [europepmc]
- Out of the laboratory and into the classroom: the future of artificial intelligence in education. [europepmc]
- Accuracy of Physicians' Electrocardiogram Interpretations: A Systematic Review and Meta-analysis. [europepmc]
- Use of an Adaptive e-Learning Platform as a Formative Assessment Tool in the Cardiovascular System Course Component of an MBBS Programme. [europepmc]
- A Natural Language Processing-Based Virtual Patient Simulator and Intelligent Tutoring System for the Clinical Diagnostic Process: Simulator Development and Case Study. [europepmc]
- Learning Analytics Applied to Clinical Diagnostic Reasoning Using a Natural Language Processing-Based Virtual Patient Simulator: Case Study. [europepmc]
- Identifying predictors of digital competence of educators and their impact on online guidance. [europepmc]
- AI-based adaptive personalized content presentation and exercises navigation for an effective and engaging E-learning platform. [europepmc]
- Intelligent tutoring systems for word problem solving in COVID-19 days: could they have been (part of) the solution? [europepmc]
- Preparing for obstetric anaesthesia-An educational randomised controlled trial comparing e-learning to written course material. [europepmc]
- The potential of learning with (and not from) artificial intelligence in education. [europepmc]
- Exploring the multidimensional relationships between educational situation perception, teacher support, online learning engagement, and academic self-efficacy in technology-based language learning. [europepmc]
- Automated Strategy Feedback Can Improve the Readability of Physicians' Electronic Communications to Simulated Patients. [europepmc]
- Adapting training in real time: An empirical test of adaptive difficulty schedules. [europepmc]
- Personalized adaptive learning in higher education: A scoping review of key characteristics and impact on academic performance and engagement. [europepmc]
- Physical embodiment and anthropomorphism of AI tutors and their role in student enjoyment and performance. [europepmc]
- A systematic review of AI-driven intelligent tutoring systems (ITS) in K-12 education. [europepmc]
- Real-time cognitive-affective dynamics of failure feedback in a technology-based learning task. [europepmc]
- The Impact of Artificial Intelligence-Supported Instruction on Student Learning in STEM: A Systematic Review and Meta-Analysis. [europepmc]
Discussions
Related