2024/02/11 by Nischal Ashok Kumar, Kumar, Nischal Ashok, Andrew Lan +1
Computer Science · #Computation and Language (cs.CL) #Educational Technology and Assessment #FOS: Computer and information sciences #Model-Driven Software Engineering Techniques #Software Engineering (cs.SE) #Software Testing and Debugging Techniques
paper · pdf · doi:10.48550/arxiv.2402.07081
openalex publication_date 2024/02/11 · openalex created_date 2024/02/14 · openalex updated_date 2026/07/28
In computer science education, test cases are an integral part of programming assignments since they can be used as assessment items to test students' programming knowledge and provide personalized feedback on student-written code. The goal of our work is to propose a fully automated approach for test case generation that can accurately measure student knowledge, which is important for two reasons. First, manually constructing test cases requires expert knowledge and is a labor-intensive process. Second, developing test cases for students, especially those who are novice programmers, is significantly different from those oriented toward professional-level software developers. Therefore, we need an automated process for test case generation to assess student knowledge and provide feedback. In this work, we propose a large language model-based approach to automatically generate test cases and show that they are good measures of student knowledge, using a publicly available dataset that contains student-written Java code. We also discuss future research directions centered on using test cases to help students.