Saying ‘no’ with confidence: statistical approaches to test for the absence of an effect

2025/10/01 by Lewis G. Halsey
Decision Sciences · Mathematics · #Meta-analysis and systematic reviews #Statistical Methods in Clinical Trials #Reliability and Agreement in Measurement

paper · doi:10.1098/rsbl.2025.0506

Abstract

Publishing non-significant findings is essential for the progress of science. However, many of us forget that ‘absence of evidence is not evidence of absence’ and believe that a statistically non-significant result is evidence of no effect. Regrettably, and despite the null hypothesis being simple, elegant and often underpinned by evidenced or reasoned convictions, conventional p -value analysis can only argue against the null hypothesis, never in favour of it. Here, I provide a quick-and-easy guide to simple yet powerful statistical options available to biologists for investigating the absence of a meaningful effect, namely equivalence tests, confidence intervals and credible intervals; or the absence of any effect, namely likelihood ratios and Bayes factors. These approaches, supported by accessible software, allow biologists to draw direct conclusions about the null hypothesis.

Citations

• The ASA Statement on p -Values: Context, Process, and Purpose
• The file drawer problem and tolerance for null results.

Related

• Why and how we should join the shift from significance testing to estimation
• Alternatives to the statistical mass confusion of testing for no-effect
• Replication of “null results” – Absence of evidence or evidence of absence?
• Replication of null results: Absence of evidence or evidence of absence?
• Alternative to the statistical mass confusion of testing for “no effect”