Making Reliability Reliable

2016/07/04 by Eunseong Cho · 1 citation
Mathematics · Decision Sciences · #Advanced Statistical Methods and Models #Psychometric Methodologies and Testing #Optimal Experimental Design Methods

paper · doi:10.1177/1094428116656239

openalex publication_date 2016/07/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/17

Abstract

The current conventions for test score reliability coefficients are unsystematic and chaotic. Reliability coefficients have long been denoted using names that are unrelated to each other, with each formula being generated through different methods, and they have been represented inconsistently. Such inconsistency prevents organizational researchers from understanding the whole picture and misleads them into using coefficient alpha unconditionally. This study provides a systematic naming convention, formula-generating methods, and methods of representing each of the reliability coefficients. This study offers an easy-to-use solution to the issue of choosing between coefficient alpha and composite reliability. This study introduces a calculator that enables its users to obtain the values of various multidimensional reliability coefficients with a few mouse clicks. This study also presents illustrative numerical examples to provide a better understanding of the characteristics and computations of reliability coefficients.

Citations

• On the Relationship between the Higher-Order Factor Model and the Hierarchical Factor Model
• The Rediscovery of Bifactor Measurement Models
• Coefficients Alpha, Beta, Omega, and the glb: Comments on Sijtsma
• From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation
• Cronbach’s α, Revelle’s β, and Mcdonald’s ωH: their Relations with Each Other and Two Alternative Conceptualizations of Reliability
• On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha
• My Current Thoughts on Coefficient Alpha and Successor Procedures
• Coefficient Alpha and the Internal Structure of Tests
• Generalizability in Factorable Domains: "Domain Validity and Generalizability"1
• Statistical Analysis of Sets of Congeneric Tests

Cited by

• Congeneric reliability [wikipedia]

Related

• A new approach to reliability assessment based on Exploratory factor analysis
• Reliability representativeness: How well does coefficient alpha summarize reliability across the score distribution?
• A Primer on Reliability via Coefficient Alpha and Omega
• Three Coefficients for Analyzing the Reliability and Validity of Ratings
• How to obtain the most error-free estimate of reliability? Eight Sources of Deflation in the Estimates of Reliability to Avoid