The Alleged Crisis and the Illusion of Exact Replication

2014/01/01 by Wolfgang Stroebe, Fritz Strack · 1 voice · 19 citations
Psychology · Social Sciences · #Behavioral Health and Interventions #Mental Health Research Topics #Social and Intergroup Psychology

paper · doi:10.1177/1745691613514450

openalex publication_date 2014/01/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/25

Abstract

There has been increasing criticism of the way psychologists conduct and analyze studies. These critiques as well as failures to replicate several high-profile studies have been used as justification to proclaim a "replication crisis" in psychology. Psychologists are encouraged to conduct more "exact" replications of published studies to assess the reproducibility of psychological research. This article argues that the alleged "crisis of replicability" is primarily due to an epistemological misunderstanding that emphasizes the phenomenon instead of its underlying mechanisms. As a consequence, a replicated phenomenon may not serve as a rigorous test of a theoretical hypothesis because identical operationalizations of variables in studies conducted at different times and with different subject populations might test different theoretical constructs. Therefore, we propose that for meaningful replications, attempts at reinstating the original circumstances are not sufficient. Instead, replicators must ascertain that conditions are realized that reflect the theoretical variable(s) manipulated (and/or measured) in the original study.

Citations

• Facilitation in recognizing pairs of words: Evidence of a dependence between retrieval operations.
• Shall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences
• Consequences of prejudice against the null hypothesis.
• False-Positive Psychology

Cited by

• Replication Studies in TESOL
• Maximal positive controls: A method for estimating the largest plausible effect size
• The case for formal methodology in scientific reform
• Experts’ reasoning about the replication crisis: Apt epistemic performance and actor-oriented transfer
• No Replication, No Trust? How Low Replicability Influences Trust in Psychology
• Making replication mainstream
• Contextual sensitivity in scientific reproducibility
• The replication crisis has led to positive structural, procedural, and community changes
• Replication studies in second language acquisition research: <i>Definitions, issues, resources, and future directions</i>
• A Validity-Based Framework for Understanding Replication in Psychology
• Noise Versus Signal: What Can One Conclude When a Classic Finding Fails to Replicate?
• Beyond Statistics: Accepting the Null Hypothesis in Mature Sciences
• A multi-study examination of intra-individual feedback loops between competence and value beliefs, procrastination, and goal achievement
• Sixty Years After Orne’s <i>American Psychologist</i> Article: A Conceptual Framework for Subjective Experiences Elicited by Demand Characteristics
• Generalizability is not optional: insights from a cross-cultural study of social discounting
• Open science, the replication crisis, and environmental public health
• On the Limitations of Manipulation Checks: An Obstacle Toward Cumulative Science
• Don’t fear ‘fear conditioning’: Methodological considerations for the design and analysis of studies on human fear acquisition, extinction, and return of fear
• Contextual Changes, Credible Conclusions? A Direct and Conceptual Replication of Shen et al.'s (2019) Study on Online Image Credibility

Discussions

• One of the classic instances of the latter is Stroebe & Strack (2014). They did not use the term "hidden moderator," as I believe that was applied later by people criticizing the argument. doi.org/10. [bsky, 5 points, 1 comments]

Related

• What is the Replication Crisis a Crisis Of?
• Replicability Crisis in Social Psychology: Looking at the Past to Find New Pathways for the Future
• Making replication mainstream
• Why most psychological research findings are not even wrong
• Why We Disagree About the Success of Replications