On Misconceptions and the Limited Usefulness of Ordinal Alpha

2017/09/21 by R. Philip Chalmers · 1 voice · 3 citations
Decision Sciences · Mathematics · #Advanced Statistical Methods and Models #Psychometric Methodologies and Testing #Reliability and Agreement in Measurement

paper · doi:10.1177/0013164417727036

openalex publication_date 2017/09/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30

Abstract

This article discusses the theoretical and practical contributions of Zumbo, Gadermann, and Zeisser's family of ordinal reliability statistics. Implications, interpretation, recommendations, and practical applications regarding their ordinal measures, particularly ordinal alpha, are discussed. General misconceptions relating to this family of ordinal reliability statistics are highlighted, and arguments for interpreting ordinal alpha as a measure of hypothetical reliability, as opposed to observed reliability, are presented. It is concluded that ordinal alpha should not be used in routine reliability analyses and reports, and instead should be understood as hypothetical tool, similar to the Spearman-Brown prophecy formula, for theoretically increasing the number of ordinal categorical response options in future applied testing applications.

Citations

• Estimating ordinal reliability for Likert-type and ordinal item response data: A conceptual, empirical, and practical guide
• Maximum Likelihood Estimation of the Polychoric Correlation Coefficient
• On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha
• Coefficient Alpha and the Internal Structure of Tests
• Creativity and its Criteria

Cited by

• A Measurement Is a Choice and Stevens’ Scales of Measurement Do Not Help Make It: A Response to Chalmers
• The Role of Item Distributions on Reliability Estimation: The Case of Cronbach’s Coefficient Alpha
• Your Coefficient Alpha Is Probably Wrong, but Which Coefficient Omega Is Right? A Tutorial on Using R to Obtain Better Reliability Estimates

Discussions

• #statstab #418 On Misconceptions and the Limited Usefulness of Ordinal Alpha Thoughts: Ordinal α seemed cool. Then I read this paper. Takeaway: methods need good peer review and debate. #ordinal #li [bsky, 4 points, 0 comments]

Related

• A Measurement Is a Choice and Stevens’ Scales of Measurement Do Not Help Make It: A Response to Chalmers
• Reliability and statistical power: Conceptual background and practical implications
• Ordinal Regression Models in Psychology: A Tutorial
• Starting at the Beginning: An Introduction to Coefficient Alpha and Internal Consistency
• Cronbach’s <i>α</i>, Revelle’s <i>β</i>, and Mcdonald’s <i>ω_H</i>: their Relations with Each Other and Two Alternative Conceptualizations of Reliability