IntegrityAI at GenAI Detection Task 2: Detecting Machine-Generated Academic Essays in English and Arabic Using ELECTRA and Stylometry

2025/01/07 by Mohammad AL-Smadi, AL-Smadi, Mohammad
Computer Science · Social Sciences · #Topic Modeling #Academic integrity and plagiarism

paper · pdf · doi:10.48550/arxiv.2501.05476

Abstract

Recent research has investigated the problem of detecting machine-generated essays for academic purposes. To address this challenge, this research utilizes pre-trained, transformer-based models fine-tuned on Arabic and English academic essays with stylometric features. Custom models based on ELECTRA for English and AraELECTRA for Arabic were trained and evaluated using a benchmark dataset. Proposed models achieved excellent results with an F1-score of 99.7%, ranking 2nd among of 26 teams in the English subtask, and 98.4%, finishing 1st out of 23 teams in the Arabic one.

Related

• GenAI Content Detection Task 2: AI vs. Human -- Academic Essay Authenticity Challenge
• AI-generated Essays: Characteristics and Implications on Automated Scoring and Academic Integrity
• The Arabic AI Fingerprint: Stylometric Analysis and Detection of Large Language Models Text
• Simple techniques to bypass GenAI text detectors: implications for inclusive education
• LuxVeri at GenAI Detection Task 1: Inverse Perplexity Weighted Ensemble for Robust Detection of AI-Generated Text across English and Multilingual Contexts