2025/08/10 by Andrew Runge, Sarah Goodwin, Yigal Attali +4 · 1 voice · 1 citation
Arts and Humanities · Psychology · Social Sciences · #EFL/ESL Teaching and Learning #Second Language Acquisition and Learning #Writing and Handwriting Education
paper · pdf · doi:10.1177/02655322251349908
openalex publication_date 2025/08/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
A longstanding criticism of traditional high-stakes writing assessments is their use of static prompts in which test takers compose a single text in response to a prompt. These static prompts do not allow measurement of the writing process. This paper describes the development and validation of an innovative interactive writing task. After the test taker responds to an initial prompt, a large language model automatically analyzes their response to identify the prompt themes they discussed. The test taker then receives a customized follow-up prompt asking them to discuss one of the themes they did not address in their original response. Several experimental results support the task’s validity. Results showed that prompt phrasing is important to elicit responses most consistent with the task expectations and that test takers are able to produce responses that are relevant. Regarding issues of fairness in administration, we found that scores are not negatively affected by the type of follow-up task. We found that an accurate theme-detection model ensures test takers are not asked to write about something they addressed in their initial response, which could threaten test score validity claims. This design brings high-stakes writing tasks into closer alignment with writing models that frame writing as a process.