A tutorial on open-source large language models for behavioral science

2024/08/15 by Zak Hussain, Marcel Binz, Rui Mata +1 · 1 voice · 12 citations
Computer Science · Social Sciences · Psychology · #Topic Modeling #Computational and Text Analysis Methods #Mental Health via Writing

paper · pdf · doi:10.3758/s13428-024-02455-8

Abstract

Large language models (LLMs) have the potential to revolutionize behavioral science by accelerating and improving the research cycle, from conceptualization to data analysis. Unlike closed-source solutions, open-source frameworks for LLMs can enable transparency, reproducibility, and adherence to data protection standards, which gives them a crucial advantage for use in behavioral science. To help researchers harness the promise of LLMs, this tutorial offers a primer on the open-source Hugging Face ecosystem and demonstrates several applications that advance conceptual and empirical work in behavioral science, including feature extraction, fine-tuning of models for prediction, and generation of behavioral responses. Executable code is made available at github.com/Zak-Hussain/LLM4BeSci.git . Finally, the tutorial discusses challenges faced by research with (open-source) LLMs related to interpretability and safety and offers a perspective on future research at the intersection of language modeling and behavioral science.

Citations

• I.—COMPUTING MACHINERY AND INTELLIGENCE
• On the Dangers of Stochastic Parrots
• ChatGPT outperforms crowd workers for text-annotation tasks

Cited by

• Automating the Scientist? Perspektiven der Fachgruppe Methoden und Evaluation auf die Nutzung von KI in der psychologischen Forschung
• Escaping the Jingle-Jangle Jungle: Increasing Conceptual Clarity in Psychology Using Large Language Models
• Sustainable Development Goals in Psychology: A Century of Progress in Publications
• Empirical Prompt Engineering for Construct Identification with Large Language Models
• Addressing Longstanding Challenges in Cognitive Science with Language Models
• Digital Prompting in Education: A Design Framework and Bibliometric Analysis
• A tentative roadmap for consensus building processes
• The Mixed Subjects Design: Treating Large Language Models as Potentially Informative Observations
• Crucible: Quantifying the Potential of Control Algorithms through LLM Agents
• Open-Source Large Language Models in Education: A Narrative Review of Evidence, Pedagogical Roles, and Learning Outcomes
• Adding LLMs to the psycholinguistic norming toolbox: A practical guide to getting the most out of human ratings
• Beyond Interpretability: Exploring the Comprehensibility of Adaptive Video Streaming through Large Language Models
• The Agent Behavior: Model, Governance and Challenges in the AI Digital Age

Discussions

• 🚨 New article 🚨 Open LLMs are powerful and reproducible alternatives to closed models like GPT or Gemini. This tutorial shows how LLMs work and how to apply open LLMs in behavioral science resear [bsky, 24 points, 0 comments]

Related

• CogBench: a large language model walks into a psychology lab
• Large language models for behavioral modeling: A literature survey
• Finetuning LLMs for Human Behavior Prediction in Social Science Experiments
• Behavioral Fingerprinting of Large Language Models
• Be.FM: Open Foundation Models for Human Behavior