REVA: Supporting LLM-Generated Programming Feedback Validation at Scale Through User Attention-based Adaptation

2025/07/15 by Xiaohang Tang, Tang, Xiaohang, Sam Wong +7
Computer Science · #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Online Learning and Analytics

paper · pdf · doi:10.48550/arxiv.2507.11470

openalex publication_date 2025/07/15 · openalex created_date 2025/10/09 · openalex updated_date 2026/07/28

Abstract

This paper introduces REVA, a human-AI system that expedites instructor review of voluminous AI-generated programming feedback by sequencing submissions to minimize cognitive context shifts and propagating instructor-driven revisions across semantically similar instances. REVA introduces a novel approach to human-AI collaboration in educational feedback by adaptively learning from instructors' attention in the review and revision process to continuously improve the feedback validation process. REVA's usefulness and effectiveness in improving feedback quality and the overall feedback review process were evaluated through a within-subjects lab study with 12 participants.

Citations

• Feedforward in Generative AI: Opportunities for a Design Space
• The Extractive-Abstractive Spectrum: Uncovering Verifiability Trade-offs in LLM Generations
• Traceable Text: Deepening Reading of AI-Generated Summaries with Phrase-Level Provenance Links
• VizGroup: An AI-Assisted Event-Driven System for Real-Time Collaborative Programming Learning Analytics
• Supporting Sensemaking of Large Language Model Outputs at Scale
• Automating Human Tutor-Style Programming Feedback: Leveraging GPT-4 Tutor Model for Hint Generation and GPT-3.5 Student Model for Hint Validation
• Beyond the Chat: Executable and Verifiable Text-Editing with LLMs
• ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing
• Using an LLM to Help With Code Understanding
• Harnessing LLMs in Curricular Design: Using GPT-4 to Support Authoring of Learning Objectives
• Generative Agents: Interactive Simulacra of Human Behavior
• AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts
• Propagating Visual Designs to Numerous Plots and Dashboards
• On the Dangers of Stochastic Parrots
• Scaffolding in Teacher–Student Interaction: A Decade of Research

Related

• The Responsible Development of Automated Student Feedback with Generative AI
• Towards Adaptive Feedback with AI: Comparing the Feedback Quality of LLMs and Teachers on Experimentation Protocols
• Humanizing Automated Programming Feedback: Fine-Tuning Generative Models with Student-Written Feedback
• Dean of LLM Tutors: Exploring Comprehensive and Automated Evaluation of LLM-generated Educational Feedback via LLM Feedback Evaluators
• Autograder+: A Multi-Faceted AI Framework for Rich Pedagogical Feedback in Programming Education