Toward best research practices in AI Psychology

2023/12/03 by Anna A. Ivanova, Ivanova, Anna A. · 2 voices
Computer Science · #Explainable Artificial Intelligence (XAI) #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.2312.01276

openalex publication_date 2023/12/03 · openalex created_date 2023/12/06 · openalex updated_date 2026/07/28

Abstract

Language models have become an essential part of the burgeoning field of AI Psychology. I discuss 14 methodological considerations that can help design more robust, generalizable studies evaluating the cognitive abilities of language-based AI systems, as well as to accurately interpret the results of these studies.

Discussions

• Running cognitive evaluations on large language models: The do's✅ and the don'ts🚫 Preprint: arxiv.org/abs/2312.01276 Thread ⬇️ [bsky, 22 points, 1 comments]
• Toward Best Research Practices in AI Psychology arxiv.org/pdf/2312.01276 [bsky, 0 points, 0 comments]

Related

• Machine Psychology
• Addressing Longstanding Challenges in Cognitive Science with Language Models
• AI Through the Human Lens: Investigating Cognitive Theories in Machine Psychology
• Large Language Models and Cognitive Science: A Comprehensive Review of Similarities, Differences, and Challenges
• A validity-guided workflow for robust large language model research in psychology