Power failure: why small sample size undermines the reliability of neuroscience

2013/04/10 by Katherine S. Button, John P. A. Ioannidis, Claire Mokrysz +5 · 3 voices · 52 citations
Computer Science · Decision Sciences · Neuroscience · #Computational Drug Discovery Methods #Functional Brain Connectivity Studies #Meta-analysis and systematic reviews

paper · pdf · doi:10.1038/nrn3475

crossref issued 2013/04/10 · crossref published 2013/04/10 · crossref published-online 2013/04/10 · openalex publication_date 2013/04/10 · crossref created 2013/04/10 · crossref published-print 2013/05/01 · crossref deposited 2023/05/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31 · crossref indexed 2026/07/31

Citations

• The Principles of Humane Experimental Technique
• Delayed reward discounting and addictive behavior: a meta-analysis
• Clarifications on the application and interpretation of the test for excess significance and its extensions
• Consequences of prejudice against the null hypothesis.
• Why Most Published Research Findings Are False
• False-Positive Psychology
• Methods in Immunology and Immunochemistry: Volume 1, Preparation of Antigens and Antibodies

Cited by

• Building a collaborative psychological science: Lessons learned from ManyBabies 1.
• Diffusion tensor imaging with free‐water correction reveals distinctions between severe and attenuated subtypes in Mucopolysaccharidosis type I
• Examining the replicability of online experiments selected by a decision market
• The Effect of an 8-Week Pilates Training Program on Flexibility and Sport-Specific Performance in Young Female Artistic Swimmers
• Dump the “dimorphism”: Comprehensive synthesis of human brain studies reveals few male-female differences beyond size
• Higher‐pitched voices are perceived as financially trustworthy
• Beyond beta rhythms: subthalamic aperiodic broadband power scales with Parkinson's disease severity–a cross-sectional multicentre study
• A standardized framework to test event-based experiments
• Why Most Acute Stroke Studies Are Positive in Animals but Not in Patients: A Systematic Comparison of Preclinical, Early Phase, and Phase 3 Clinical Trials of Neuroprotective Agents
• How many trials does it take to get a significant ERP effect? It depends
• Reproducible brain-wide association studies require thousands of individuals
• Influences on Data Quality in Developmental Children Studies
• Deep impact: unintended consequences of journal rank
• This human study did not involve human subjects: Validating LLM simulations as behavioral evidence
• Translational network neuroscience: Nine roadblocks and possible solutions
• Open minds, tied hands: Awareness, behavior, and reasoning on open science and irresponsible research behavior
• Zapping the brain to enhance sport performance? An umbrella review of the effect of transcranial direct current stimulation on physical performance
• On effects that do occur versus effects that can be made to occur
• Big little lies: a compendium and simulation of <i>p</i> -hacking strategies
• Odor cueing of declarative memories during sleep enhances coordinated spindles and slow oscillations
• A reproducible and generalizable software workflow for analysis of large-scale neuroimaging data collections using BIDS Apps
• The past, present and future of neuroscience data sharing: a perspective on the state of practices and infrastructure for FAIR
• “Nothing to see here”: No structural brain differences as a function of the<i>Big Five</i>personality traits from a systematic review and meta-analysis
• Developmental psychologists should adopt citizen science to improve generalization and reproducibility
• MYND: Unsupervised Evaluation of Novel BCI Control Strategies on\n Consumer Hardware
• Heartbeat-evoked responses in M/EEG: A systematic review of methods with suggestions for analysis and reporting
• Computational complexity as a potential limitation on brain–behaviour mapping
• Teoría y práctica de la calidad
• An evaluation of reproducibility and errors in published sample size calculations performed using G*Power
• Virtual reality enhanced mindfulness and yoga intervention for postpartum depression and anxiety in the post COVID era
• Understanding the current evidence base for the commonly recommended management strategies for recurrent feline idiopathic cystitis: a systematic review
• Methodological considerations for studying neural oscillations
• Time to adjust: Improving replicability in experimental psychology by adjustment for evident selective inference
• A framework for assessing the trustworthiness of scientific research findings
• Amygdala-centered mechanisms of pain amplification and chronification in fibromyalgia and migraine: Narrative review
• Enhancing precision in human neuroscience
• What Is the Test-Retest Reliability of Common Task-Functional MRI Measures? New Empirical Evidence and a Meta-Analysis
• Replication value as a function of citation impact and sample size
• The Psychological Science Accelerator: Advancing Psychology Through a Distributed Collaborative Network
• The Hype Cycle of Working Memory Training
• Power Calculations for Replication Studies
• Efficacy and working mechanisms of a Go/No-Go task-based inhibition training in smoking: A randomized-controlled trial
• Parental education and income are linked to offspring cortical brain structure and psychopathology at 9–11 years
• Publication bias impacts on effect size, statistical power, and magnitude (Type M) and sign (Type S) errors in ecology and evolutionary biology
• In search of the lost interaction: A theoretical and methodological framework for researching interactions
• Multiband acquisition sequences for fMRI: Proceed with caution
• With Little Power Comes Great Responsibility
• Moving beyond noninformative priors: why and how to choose weakly informative priors in Bayesian analyses
• A solution to minimum sample size for regressions
• Small Telescopes
• Exploring the Role of Randomization on Belief Rigidity in Online Social Networks
• Transparency and reproducibility in invasion science
• On doing relevant and rigorous experiments: Review and recommendations
• Impact of genetic alterations on central nervous system progression of primary vitreoretinal lymphoma

Discussions

• It would be a critique of 'field-wide power analysis' of the sort done by Button et al. (2013) - a (terrible) paper that's been cited >10000 times doi.org/10.1038/nrn3... [bsky, 1 points, 1 comments]
• Why? This is beautifully visualized in Fig. 1b of this classic paper: doi.org/10.1038/nrn3... In our recent publication, we show how scaling up to 119 patients across 4 hospitals strengthens biomark [bsky, 0 points, 1 comments]
• 1. The paper & its claims: doi.org/10.1038/nrn3... Examined 49 #meta-analyses (MA) published in 2011, across "neuroscience" These 49 MA included 730 original studies Median effect-size & sample siz [bsky, 0 points, 2 comments]