Reaction time in differential and developmental research: A review and commentary on the problems and alternatives.

2019/03/21 by Christopher Draheim, Cody A. Mashburn, Jessie D. Martin +3 · 339 citations
Psychology · #Affect (linguistics) #Applied psychology #Artificial intelligence #Behavioral Health and Interventions #Cognition #Cognitive Abilities and Testing #Cognitive psychology #Computer science #Empirical research #Interpretability #MEDLINE #Mental Health Research Topics #Popularity #PsycINFO #Psychological research #Psychology #Reliability (semiconductor) #Social psychology #Statistics #Task (project management)

paper · doi:10.1037/bul0000192

published in Psychological Bulletin 145(5), 508-535 (American Psychological Association)

openalex publication_date 2019/03/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30

Abstract

Reaction time is believed to be a good indicator of the speed and efficiency of mental processes and is a ubiquitous variable in the behavioral sciences. Despite this popularity, there are numerous issues associated with using reaction time (RT), specifically in differential and developmental research. Here, we identify and focus on two main problems-unreliability and sensitivity to speed-accuracy interactions. The use of difference scores is a primary factor that leads to many RT measures having demonstrably low reliability, and RT measures in general often do not properly account for speed-accuracy interactions. Both factors jeopardize the validity and interpretability of results based on RT. Here, we evaluate conceptually and empirically how these issues affect individual differences research. Although the empirical evidence we provide are primarily within the domains of attention control and task switching, we highlight examples from various other areas of psychological inquiry. We also discuss many of the statistical and methodological alternatives available to researchers conducting correlational studies. Ultimately, we encourage researchers comparing individuals of differing cognitive and developmental levels to strongly consider using these alternatives in lieu of RT, specifically RT difference scores. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Citations

• Adaptive procedures in psychophysical research
• Effects of noise letters upon the identification of a target letter in a nonsearch task
• Studies of interference in serial verbal reactions.

Cited by

• Measuring the acuity of the approximate number system in young children
• Assessing the validity evidence for habit measures based on time pressure
• Dynamic changes in interpretation bias as predictors of social anxiety symptom change during CBT
• Differential attentional demands on implicit and explicit associative memory in children 8–12 years old
• Baseline Pupil Size Seems Unrelated to Fluid Intelligence, Working Memory Capacity, and Attentional Control
• On the reliability of value-modulated attentional capture: An online replication and multiverse analysis
• Testing the relationship between preferences for infant-directed speech and vocabulary development: A multi-lab study
• Process-oriented intelligence research: A review from the cognitive perspective
• A toolbox approach to improving the measurement of attention control.
• Individual differences in computational psychiatry: A review of current challenges
• Attentional bias in alcohol drinkers: A systematic review of its link with consumption variables
• Increasing Computation Resolves Conflicts in Vision Language Models
• A new way of classifying developmental prosopagnosia: Balanced Integration Score
• Interference control in working memory: Evidence for discriminant validity between removal and inhibition tasks. [europepmc]
• Self-reported impulsivity does not predict response caution. [europepmc]
• Training Using a Commercial Immersive Virtual Reality System on Hand-Eye Coordination and Reaction Time in Young Musicians: A Pilot Study. [europepmc]
• The stability and reliability of attentional bias measures in the dot-probe task: Evidence from both traditional mean bias scores and trial-level bias scores. [europepmc]
• Individual Differences in Inhibitory Control: A latent Variable Analysis. [europepmc]
• Cognitive efficiency beats top-down control as a reliable individual difference dimension relevant to self-control. [europepmc]
• Best research practices for using the Implicit Association Test. [europepmc]
• Assessing the three attentional networks in children from three to six years: A child-friendly version of the Attentional Network Test for Interaction. [europepmc]
• The role of attention control in complex real-world tasks. [europepmc]
• The Effects of Proprioceptive Training on Balance, Strength, Agility and Dribbling in Adolescent Male Soccer Players. [europepmc]
• Task-general efficiency of evidence accumulation as a computationally-defined neurocognitive trait: Implications for clinical neuroscience. [europepmc]
• Individual differences in artificial and natural language statistical learning. [europepmc]
• Same same but different: Subtle but consequential differences between two measures to linearly integrate speed and accuracy (LISAS vs. BIS). [europepmc]
• Linking computational models of two core tasks of cognitive control. [europepmc]
• Resting-state BOLD signal variability is associated with individual differences in metacontrol. [europepmc]
• A review of attentional bias modification trainings for depression. [europepmc]
• Improving the Reliability of Cognitive Task Measures: A Narrative Review. [europepmc]
• On the psychometric evaluation of cognitive control tasks: An Investigation with the Dual Mechanisms of Cognitive Control (DMCC) battery. [europepmc]
• Calibration of cognitive tests to address the reliability paradox for decision-conflict tasks. [europepmc]
• Testing locus coeruleus-norepinephrine accounts of working memory, attention control, and fluid intelligence. [europepmc]
• Replication and extension of the toolbox approach to measuring attention control. [europepmc]
• Why many studies of individual differences with inhibition tasks may not localize correlations. [europepmc]
• On the reliability of value-modulated attentional capture: An online replication and multiverse analysis. [europepmc]
• The common factor of executive functions measures nothing but speed of information uptake. [europepmc]
• Using precision approaches to improve brain-behavior prediction. [europepmc]

Related

• We don't know what you did last summer. On the importance of transparent reporting of reaction time data pre-processing
• A toolbox approach to improving the measurement of attention control.
• Age Differences in Intra-Individual Variability in Simple and Choice Reaction Time: Systematic Review and Meta-Analysis
• How Participants Respond to Computer Delays
• The thermodynamics of human reaction times