2023/09/07 by Claudia Biancotti, Biancotti, Claudia, Carolina Camassa +1
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #J.4 #K.4.1 #Text Readability and Simplification
paper · pdf · doi:10.48550/arxiv.2309.03595
openalex publication_date 2023/09/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
ChatGPT, a software seeking to simulate human conversational abilities, is attracting increasing attention. It is sometimes portrayed as a groundbreaking productivity aid, including for creative work. In this paper, we run an experiment to assess its potential in complex writing tasks. We ask the software to compose a policy brief for the Board of the Bank of Italy. We find that ChatGPT can accelerate workflows by providing well-structured content suggestions, and by producing extensive, linguistically correct text in a matter of seconds. It does, however, require a significant amount of expert supervision, which partially offsets productivity gains. If the app is used naively, output can be incorrect, superficial, or irrelevant. Superficiality is an especially problematic limitation in the context of policy advice intended for high-level audiences.