Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessment

2025/07/07 by Pei, Jiahuan, Ye, Fanghua, Sun, Xin +3 · 3 citations
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2507.05528

Abstract

Large language models (LLMs) have advanced virtual educators and learners, bridging NLP with AI4Education. Existing work often lacks scalability and fails to leverage diverse, large-scale course content, with limited frameworks for assessing pedagogic quality. To this end, we propose WikiHowAgent, a multi-agent workflow leveraging LLMs to simulate interactive teaching-learning conversations. It integrates teacher and learner agents, an interaction manager, and an evaluator to facilitate procedural learning and assess pedagogic quality. We introduce a dataset of 114,296 teacher-learner conversations grounded in 14,287 tutorials across 17 domains and 727 topics. Our evaluation protocol combines computational and rubric-based metrics with human judgment alignment. Results demonstrate the workflow's effectiveness in diverse setups, offering insights into LLM capabilities across domains. Our datasets and implementations are fully open-sourced.

Cited by

• ClinTutor-R1: Advancing Scalable and Robust One-to-Many Alignment in Clinical Socratic Education
• Trustworthy AI Psychotherapy: Multi-Agent LLM Workflow for Counseling and Explainable Mental Disorder Diagnosis
• Opportunities and Challenges of LLMs in Education: An NLP Perspective

Related

• EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework
• LLM Agents for Education: Advances and Applications
• TeachLM: Post-Training LLMs for Education Using Authentic Learning Data
• Dialogic Pedagogy for Large Language Models: Aligning Conversational AI with Proven Theories of Learning
• Enabling Multi-Agent Systems as Learning Designers: Applying Learning Sciences to AI Instructional Design