2025/10/10 by Alessandro De Cassai · 1 voice
Decision Sciences · Economics, Econometrics and Finance · Medicine · #Cardiac, Anesthesia and Surgical Outcomes #Health Systems, Economic Evaluations, Quality of Life #Meta-analysis and systematic reviews
paper · pdf · doi:10.1111/anae.70028
openalex publication_date 2025/10/10 · openalex created_date 2025/10/11 · openalex updated_date 2026/07/15
I read with interest the letter by Rovetta et al. [1] in response to Huber [2]. As they highlight, all values within a confidence interval are compatible equally with the true effect. However, convention dictates that overlap with the line of no effect, or a p value ≥ 0.05, should be interpreted as ‘no significant effect’. Such a dichotomy can be misleading, as a wide confidence interval lying predominantly in the direction of benefit (or harm) may still be dismissed, simply because its extreme value touches the null value [1]. Equally, statistical significance does not guarantee clinical relevance. An effect must be judged by whether it differs from zero but also by whether it exceeds a threshold of benefit meaningful to patients and clinicians. This threshold is usually defined as the minimal clinically important difference [3]. I propose a simple extension to routine reporting, which is to combine the confidence interval with the minimal clinically important difference to indicate how much of the plausible effect distribution lies beyond the chosen threshold. Let Δe be the estimated difference, ΔL and ΔU the confidence interval boundaries and δ the minimal clinically important difference. For continuous outcomes, the effect is clearly clinically beneficial if ΔL > δ and clearly not beneficial if ΔU < δ. When the confidence interval overlaps with the minimal clinically important difference (ΔL < δ < ΔU), one may calculate the proportion of the interval compatible with benefit as Pδ = (ΔU − δ)/(ΔU − ΔL), with 0 ≤ Pδ ≤ 1. Values close to 1 imply that most of the interval lies above the minimal clinically important difference. Values close to 0 imply that little of it does. Values close to 0.5 indicate uncertainty. For ratio measures, such as relative risks or odds ratios, the same reasoning applies but on the logarithmic scale. Let θe be the log estimate, θL to θU the confidence interval and θδ = log(δ). If θU < θδ, the intervention can be regarded as effective. If θL > θδ, it is not effective. Otherwise, the proportion of the interval beyond the minimal clinically important difference can be expressed as (θδ – θL)/(θU – θL). Consider a meta-analysis reporting an OR (95% CI) of 0.70 (0.49–1.01). On the log scale, this interval is approximately -0.71 to 0.01. If the chosen minimal clinically important difference is an OR of 0.85 (log value -0.162), then the calculation gives (-0.162-(-0.713))/(0-(-0.713)) = 0.76. Hence, approximately three-quarters of the interval supports a benefit beyond the minimal clinically important difference, even though conventional rules would declare the result as not statistically significant. Such an approach would not replace the standard reporting of point estimates, confidence intervals and p values, but would link statistical uncertainty explicitly to clinical relevance. It would also encourage authors to state and justify their chosen minimal clinically important difference, which is too often absent. However, the Pδ must not be mistaken for the probability that the true effect exceeds this threshold. Such probabilistic statements require a Bayesian framework. The measure is sensitive to the width of the interval and the confidence level chosen (e.g. 90% vs. 95%). Moreover, any decision threshold, such as treating values > 0.5 as clinically relevant, is arbitrary. Despite these caveats, expressing how much of a confidence interval lies beyond the minimal clinically important difference may help move us away from rigid reliance on p values and towards a more clinically grounded interpretation of trial evidence.