Pedagogy-R1: Pedagogically-Aligned Reasoning Model with Balanced Educational Benchmark

2025/05/24 by Lee, Unggi, Lee, Jaeyong, Bae, Jiyeong +6
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2505.18467

Abstract

Recent advances in large reasoning models (LRMs) show strong performance in structured domains such as mathematics and programming; however, they often lack pedagogical coherence and realistic teaching behaviors. To bridge this gap, we introduce Pedagogy-R1, a framework that adapts LRMs for classroom use through three innovations: (1) a distillation-based pipeline that filters and refines model outputs for instruction-tuning, (2) the Well-balanced Educational Benchmark (WBEB), which evaluates performance across subject knowledge, pedagogical knowledge, tracing, essay scoring, and teacher decision-making, and (3) a Chain-of-Pedagogy (CoP) prompting strategy for generating and eliciting teacher-style reasoning. Our mixed-method evaluation combines quantitative metrics with qualitative analysis, providing the first systematic assessment of LRMs' pedagogical strengths and limitations.

Citations

• SEAL: Steerable Reasoning Calibration of Large Language Models for Free
• Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyond
• An Empirical Study on Eliciting and Improving R1-like Reasoning Models
• MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors
• From System 1 to System 2: A Survey of Reasoning Large Language Models
• Limitations of Large Language Models in Clinical Problem-Solving Arising from Inflexible Reasoning
• Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs
• Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems
• Enhancing LLM Reasoning with Reward-guided Tree Search
• Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure
• CLST: Cold-Start Mitigation in Knowledge Tracing by Aligning a Generative Language Model as a Students' Knowledge Tracer
• Language Model Can Do Knowledge Tracing: Simple but Effective Method to Integrate Language Model and Knowledge Tracing Task
• Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach
• Large Language Models for Education: A Survey and Outlook
• Language Models as Science Tutors
• Difficulty-Focused Contrastive Learning for Knowledge Tracing with a Large Language Model-Based Difficulty Prediction
• LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis
• ChatGPT for good? On opportunities and challenges of large language models for education
• The NCTE Transcripts: A Dataset of Elementary Math Classroom Transcripts
• Computational Grounded Theory: A Methodological Framework
• DBE-KT22: A Knowledge Tracing Dataset Based on Online Student Evaluation

Related

• TPD: Enhancing Student Language Model Reasoning via Principle Discovery and Guidance
• MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors
• Trace-of-Thought Prompting: Investigating Prompt-Based Knowledge Distillation Through Question Decomposition
• Learning from Committee: Reasoning Distillation from a Mixture of Teachers with Peer-Review
• EduFlow: Advancing MLLMs' Problem-Solving Proficiency through Multi-Stage, Multi-Perspective Critique