The Practical Alternative to the p Value Is the Correctly Used p Value

2021/02/09 by Daniël Lakens · 1 voice · 155 citations
Computer Science · Decision Sciences · Mathematics · Psychology · #Alternative hypothesis #Ask price #Commit #Computer science #Data Analysis with R #Epistemology #Fallacy #Mathematics #Meta-analysis and systematic reviews #Null hypothesis #Psychology #Quality (philosophy) #Statistic #Statistical hypothesis testing #Statistician #Statistics #Toolbox #Value (mathematics) #p-value

paper · pdf · doi:10.1177/1745691620958012

published in Perspectives on Psychological Science 16(3), 639-648 (SAGE Publishing)

openalex publication_date 2021/02/09 · openalex created_date 2021/02/15 · openalex updated_date 2026/08/01

Abstract

Because of the strong overreliance on p values in the scientific literature, some researchers have argued that we need to move beyond p values and embrace practical alternatives. When proposing alternatives to p values statisticians often commit the “statistician’s fallacy,” whereby they declare which statistic researchers really “want to know.” Instead of telling researchers what they want to know, statisticians should teach researchers which questions they can ask. In some situations, the answer to the question they are most interested in will be the p value. As long as null-hypothesis tests have been criticized, researchers have suggested including minimum-effect tests and equivalence tests in our statistical toolbox, and these tests have the potential to greatly improve the questions researchers ask. If anyone believes p values affect the quality of scientific research, preventing the misinterpretation of p values by developing better evidence-based education and user-centered statistical software should be a top priority. Polarized discussions about which statistic scientists should use has distracted us from examining more important questions, such as asking researchers what they want to know when they conduct scientific research. Before we can improve our statistical inferences, we need to improve our statistical questions.

Citations

• Equivalence Testing for Psychological Research: A Tutorial
• Moving to a World Beyond “ p < 0.05”

Cited by

• The epistemic and pragmatic function of dichotomous claims based on statistical hypothesis tests
• The Recurrence of Fundamental Questions: A Historical and Philosophical Analysis of Major Disciplinary Crises in Psychology
• Nonsignificance misinterpreted as an effect’s absence in psychology: prevalence and temporal analyses
• The Benefits of Reporting Critical-Effect-Size Values
• In search of the lost interaction: A theoretical and methodological framework for researching interactions
• Towards better hypothesis tests in oxytocin research: Evaluating the validity of auxiliary assumptions
• Use and misuse of corrections for multiple testing
• Psychology Is a Property of Persons, Not Averages or Distributions: Confronting the Group-to-Person Generalizability Problem in Experimental Psychology
• Diagnosing the Misuse of the Bayes Factor in Applied Research
• Ten common statistical mistakes to watch out for when writing or reviewing a manuscript
• Current Status of Open Science and Statistical Analysis in <i>The Japanese Journal of Educational Psychology</i>:
• Ten common statistical mistakes to watch out for when writing or reviewing a manuscript. [europepmc]
• Verifying molecular clusters by 2-color localization microscopy and significance testing. [europepmc]
• Statistical Inferences Using Effect Sizes in Human Endothelial Function Research. [europepmc]
• Applied Bayesian Approaches for Research in Motor Neuron Disease. [europepmc]
• Money Does Not Always Buy Happiness, but Are Richer People Less Happy in Their Daily Lives? It Depends on How You Analyze Income. [europepmc]
• Interaction effect: Are you doing the right thing? [europepmc]
• Questionable Research Practices, Low Statistical Power, and Other Obstacles to Replicability: Why Preclinical Neuroscience Research Would Benefit from Registered Reports. [europepmc]
• In Defense of Hypothesis Testing: A Response to the Joint Editorial From the International Society of Physiotherapy Journal Editors on Statistical Inference Through Estimation. [europepmc]
• Don't Be Fooled by Randomness: Valid p -Values for Single Molecule Microscopy. [europepmc]
• Neuroscience Needs to Test Both Statistical and Scientific Hypotheses. [europepmc]
• Better Inference in Neuroscience: Test Less, Estimate More. [europepmc]
• Replication concerns in sports and exercise science: a narrative review of selected methodological issues in the field. [europepmc]
• Assumption-checking rather than (just) testing: The importance of visualization and effect size in statistical diagnostics. [europepmc]
• Testing delayed, gradual, and temporary treatment effects in randomized single-case experiments: A general response function framework. [europepmc]
• Exploring a multimodal approach for utilizing digital biomarkers for childhood mental health screening. [europepmc]
• Statistical Significance versus Clinical Relevance: Key Considerations in Interpretation Medical Research Data. [europepmc]
• Nonsignificance misinterpreted as an effect's absence in psychology: prevalence and temporal analyses. [europepmc]
• Machine-learning-based prediction of functional recovery in deep-pain-negative dogs after decompressive thoracolumbar hemilaminectomy for acute intervertebral disc extrusion. [europepmc]
• Approaching Significance: Statistical Guidance for Authors and Reviewers. [europepmc]
• Robust statistical methods and the credibility movement of psychological science. [europepmc]
• Peripheral cyclical expression of GABA-A receptor subunit genes and menstrual cycle affective change: a dimensional, transdiagnostic study. [europepmc]
• Design and analysis of behavioral intervention studies: A Bayesian approach. [europepmc]
• In Search of the Lost Interaction: A Theoretical and Methodological Framework for Researching Interactions. [europepmc]
• The Contribution of Bayesian Methods in Solving the Paradoxes of Classical Statistical Tests in Biomedical Research. [europepmc]
• Neural Efficiency and Attentional Instability in Gaming Disorder: A Task-Based Occipital EEG and Machine Learning Study [europepmc]

Discussions

• Some people think that 'the null is never true' is a killer argument against p values. It isn't. If you believe this, test against a meaningful null - e.g., r > 0.1, instead of r > 0. Moving to range [bsky, 24 points, 0 comments]

Related

• P Value and the Theory of Hypothesis Testing: An Explanation for New Researchers
• Why P Values Are Not a Useful Measure of Evidence in Statistical Significance Testing
• Thou Shalt Not Reject the P-value
• Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
• Valid P -Values Behave Exactly as They Should: Some Misleading Criticisms of P -Values and Their Resolution With S -Values