The Abuse of Power
2001/02/01 by John M Hoenig, John M. Hoenig, Dennis M Heisey +1 · 62 citations
Mathematics · Decision Sciences · Economics, Econometrics and Finance · #Statistical Methods in Clinical Trials #Meta-analysis and systematic reviews #Health Systems, Economic Evaluations, Quality of Life
paper · doi:10.1198/000313001300339897
Abstract
AbstractIt is well known that statistical power calculations can be valuable in planning an experiment. There is also a large literature advocating that power calculations be made whenever one performs a statistical test of a hypothesis and one obtains a statistically nonsignificant result. Advocates of such post-experiment power calculations claim the calculations should be used to aid in the interpretation of the experimental results. This approach, which appears in various forms, is fundamentally flawed. We document that the problem is extensive and present arguments to demonstrate the flaw in the logic.KEY WORDS: Bioequivalence testingBurden of proofObserved powerRetrospective power analysisStatistical powerType II error
Cited by
- The ASA Statement on p -Values: Context, Process, and Purpose
- Some Practical Guidelines for Effective Sample Size Determination
- Can Citizen Pressure Be Induced to Improve Public Service Provision?
- Distance‐Based Tests for Homogeneity of Multivariate Dispersions
- New Author Guidelines for Displaying Data and Reporting Data Analysis and Statistical Methods in Experimental Biology
- Recommendations to improve use and reporting of statistics in animal experiments
- Post Hoc Power: Not Empowering, Just Misleading
- Statistical Power in Plant Pathology Research
- Toward a More Powerful Experimental Communication Science: An Assessment of Two Decades’ Research (2001–2023)
- Internalizing and externalizing symptoms and aggression and violence in men and women
- Effect size estimates: Current use, calculations, and interpretation.
- Why and how we should join the shift from significance testing to estimation
- No evidence for deleterious effects of psilocybin and MDMA on working memory or related neurophysiological activity in a non-clinical population: An uncontrolled exploratory study
- Bayes Factor Hypothesis Testing in Meta-Analyses: Practical Advantages and Methodological Considerations
- Replication of “null results” – Absence of evidence or evidence of absence?
- Bayesian estimation supersedes the t test.
- Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
- A Reckless Guide to P-values
- Fare evasion and information provision: What information should be provided to reduce fare-evasion?
- Sample size determination and power analysis using the G*Power software
- I can see clearly now: reinterpreting statistical significance
- The statistical significance filter leads to overoptimistic expectations of replicability
- How Many Participants Do I Need to Test an Interaction? Conducting an Appropriate Power Analysis and Achieving Sufficient Power to Detect an Interaction
- Navigating the Social Environment: Linking Motivations, Impressions, and Behaviors Via a Sampling Approach to Trustworthiness
- Power analysis for random‐effects meta‐analysis
- Variation in reproductive photosynthetic compensation of distinct germplasm varieties of a native rangeland grass, <i>Pseudoroegneria spicata,</i> following floral defoliation
- Replication of null results: Absence of evidence or evidence of absence?
- Replication, statistical consistency, and publication bias
- Sample Size Planning for Statistical Power and Accuracy in Parameter Estimation
- Couple similarity in empathic accuracy and relationship well-being
- Increased anticholinergic medication use in middle‐aged and older autistic adults and its associations with self‐reported memory difficulties and cognitive decline
- Relational Savoring as an Approach to Improve Marital Relationship Outcomes: A Pilot Study Among Iranian Wives
- With Little Power Comes Great Responsibility
- Exploring the role of future hypothetical prompts in children’s planning abilities
- To P or not to P: on the evidential nature of P-values and their place in scientific inference
- Does organic farming jeopardize food security of farm households in Benin?
- Publication bias, statistical power and reporting practices in the Journal of Sports Sciences: potential barriers to replicability
- Providing education on evidence-based practice improved knowledge but did not change behaviour: a before and after study. [europepmc]
- Statistics in brief: the importance of sample size in the planning and interpretation of medical research. [europepmc]
- Current sample size conventions: flaws, harms, and alternatives. [europepmc]
- Transgenerational consequences of PTSD: risk factors for the mental health of children whose mothers have been exposed to the Rwandan genocide. [europepmc]
- Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. [europepmc]
- Definitional Differences of 'Outpatient' Versus 'Inpatient' THA and TKA Can Affect Study Outcomes. [europepmc]
- Cerebrovascular and amyloid pathology in predementia stages: the relationship with neurodegeneration and cognitive decline. [europepmc]
- Genetic alterations driving metastatic colony formation are acquired outside of the primary tumour in melanoma. [europepmc]
- Bayesian alternatives for common null-hypothesis significance tests in psychiatry: a non-technical guide using JASP. [europepmc]
- Ecological Momentary Assessment Is a Feasible and Valid Methodological Tool to Measure Older Adults' Physical Activity and Sedentary Behavior. [europepmc]
- Reporting of stepped wedge cluster randomised trials: extension of the CONSORT 2010 statement with explanation and elaboration. [europepmc]
- Improving transparency and scientific rigor in academic publishing. [europepmc]
- Age differences in everyday stressor-related negative affect: A coordinated analysis. [europepmc]
- How Many Participants Do We Have to Include in Properly Powered Experiments? A Tutorial of Power Analysis with Reference Tables. [europepmc]
- Global mycorrhizal plant distribution linked to terrestrial carbon stocks. [europepmc]
- Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools. [europepmc]
- Estimating power in (generalized) linear mixed models: An open introduction and tutorial in R. [europepmc]
- Recommendations for Statistical Reporting in Cardiovascular Medicine: A Special Report From the American Heart Association. [europepmc]
- Sample size determination and power analysis using the G*Power software. [europepmc]
- Post hoc power is not informative. [europepmc]
- High-Dose Dexamethasone and Oxygen Support Strategies in Intensive Care Unit Patients With Severe COVID-19 Acute Hypoxemic Respiratory Failure: The COVIDICUS Randomized Clinical Trial. [europepmc]
- Post-COVID-19 Conditions Among Children 90 Days After SARS-CoV-2 Infection. [europepmc]
- The relationship between leisure activities and mental health: The impact of resilience and COVID-19. [europepmc]
- Change in gait speed and fall risk among community-dwelling older adults with and without mild cognitive impairment: a retrospective cohort analysis. [europepmc]
- Power to Detect What? Considerations for Planning and Evaluating Sample Size. [europepmc]
Related