2024/08/03 by Gauri Anil Godghase, Rishit Agrawal, Godghase, Gauri Anil +5 · 1 citation
Computer Science · #AI in Service Interactions #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Sentiment Analysis and Opinion Mining
paper · pdf · doi:10.48550/arxiv.2408.04647
openalex publication_date 2024/08/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
There have been many recent advances in the fields of generative Artificial Intelligence (AI) and Large Language Models (LLM), with the Generative Pre-trained Transformer (GPT) model being a leading "chatbot." LLM-based chatbots have become so powerful that it may seem difficult to differentiate between human-written and machine-generated text. To analyze this problem, we have developed a new dataset consisting of more than 750,000 human-written paragraphs, with a corresponding chatbot-generated paragraph for each. Based on this dataset, we apply Machine Learning (ML) techniques to determine the origin of text (human or chatbot). Specifically, we consider two methodologies for tackling this issue: feature analysis and embeddings. Our feature analysis approach involves extracting a collection of features from the text for classification. We also explore the use of contextual embeddings and transformer-based architectures to train classification models. Our proposed solutions offer high classification accuracy and serve as useful tools for textual analysis, resulting in a better understanding of chatbot-generated text in this era of advanced AI technology.