Scientific Utopia

2012/11/01 by Brian A. Nosek, Jeffrey R. Spies, Matt Motyl · 1 voice · 24 citations
Arts and Humanities · Decision Sciences · #Meta-analysis and systematic reviews #Philosophy and History of Science #scientometrics and bibliometrics research

paper · doi:10.1177/1745691612459058

openalex publication_date 2012/11/01 · openalex created_date 2021/02/01 · openalex updated_date 2026/08/04

Abstract

An academic scientist's professional success depends on publishing. Publishing norms emphasize novel, positive results. As such, disciplinary incentives encourage design, analysis, and reporting decisions that elicit positive results and ignore negative results. Prior reports demonstrate how these incentives inflate the rate of false effects in published science. When incentives favor novelty over replication, false results persist in the literature unchallenged, reducing efficiency in knowledge accumulation. Previous suggestions to address this problem are unlikely to be effective. For example, a journal of negative results publishes otherwise unpublishable reports. This enshrines the low status of the journal and its content. The persistence of false findings can be meliorated with strategies that make the fundamental but abstract accuracy motive-getting it right-competitive with the more tangible and concrete incentive-getting it published. This article develops strategies for improving scientific practices and knowledge accumulation that account for ordinary human motivations and biases.

Citations

• The file drawer problem and tolerance for null results.
• Shall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences
• Patterns of evaluation in science: Institutionalisation, structure and functions of the referee system
• A power primer.
• Exploring Scientific Misconduct: Isolated Individuals, Impure Institutions, or an Inevitable Idiom of Modern Science?
• Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
• Creating false memories: Remembering words not presented in lists.
• Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
• Peer-review practices of psychological journals: The fate of published articles, submitted again
• Statistical power analysis for the behavioral sciences
• Consequences of prejudice against the null hypothesis.
• Scientific Utopia: I. Opening Scientific Communication
• Age differences in veridical and false recall are not inevitable: The role of frontal lobe function
• Why Most Published Research Findings Are False
• The case for motivated reasoning.
• False-Positive Psychology
• Temporal construal.
• Two new Later Stone Age sites from the Final Pleistocene in the Falémé Valley, eastern Senegal

Cited by

• Misconduct and Questionable Research Practices: The Ethics of Quantitative Data Handling and Reporting in Applied Linguistics
• The research we don’t need will persist until we dismantle the systems that sustain it
• Changing institutional incentives to foster sound scientific practices: One department
• Assessing scientists for hiring, promotion, and tenure
• Investigating the replicability of preclinical cancer biology
• Show Me the Money: A Systematic Exploration of Manipulations, Moderators, and Mechanisms of Priming Effects
• Challenges for assessing replicability in preclinical cancer biology
• The case for formal methodology in scientific reform
• Facts Are More Important Than Novelty
• Underreporting in Psychology Experiments
• Open science challenges, benefits and tips in early career and beyond
• Replicability and Reproducibility in Comparative Psychology
• Raising Awareness for the Replication Crisis in Clinical Psychology by Focusing on Inconsistencies in Psychotherapy Research: How Much Can We Rely on Published Findings from Efficacy Trials?
• ON NORMATIVITY AND EPISTEMIC INTUITIONS: FAILURE OF REPLICATION
• Top 10 Replicated Findings From Behavioral Genetics
• Is the association between children’s baby video viewing and poor language development robust? A reanalysis of Zimmerman, Christakis, and Meltzoff (2007).
• An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science
• A Review on Blockchain Technology and Blockchain Projects Fostering Open Science
• Effects of charismatic leadership and rewards on individual performance
• The preregistration revolution
• The prevalence of statistical reporting errors in psychology (1985–2013)
• The Extent and Consequences of P-Hacking in Science
• The when and how of planning: Meta-analysis of the scope and components of implementation intentions in 642 tests
• What helps and hinders reproducible research? Researchers’ perspectives from a cross-disciplinary interview study

Discussions

• "Knowledge building incentives are satisfied when the research has impact on new investigations. By reducing the value of publication, the comparative value of having impact on other research increas [bsky, 0 points, 0 comments]

Related

• Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability
• The natural selection of good science
• Ending publication bias: A values-based approach to surface null and negative results
• Publication bias and the canonization of false facts
• The misalignment of incentives in academic publishing and implications for journal reform