Three Coefficients for Analyzing the Reliability and Validity of Ratings

1985/03/01 by Lewis R. Aiken · 1,298 citations
Decision Sciences · Mathematics · Social Sciences · #Consistency (knowledge bases) #Delphi Technique in Research #Discrete mathematics #Health Education and Validation #Internal consistency #Mathematics #Population #Psychometric Methodologies and Testing #Psychometrics #Reliability (semiconductor) #Standard deviation #Statistics

paper · doi:10.1177/0013164485451012

published in Educational and Psychological Measurement 45(1), 131-142 (SAGE Publishing)

openalex publication_date 1985/03/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Three numerical coefficients (V, R, and H) for analyzing the validity and reliability of ratings are described. Each coefficient, which ranges in value from 0 to 1, is computed as the ratio of an obtained to a maximum sum of differences in ratings, or as 1 minus that ratio. Computer programs for calculating the coefficients, their associated individual and cumulative right-tail probabilities, and the population mean and standard deviation of each coefficient are available. Tables of the individual and right-tail probabilities for selected values of the three coefficients are included for 2 to 7 rating categories and 2 to 25 items or raters. More complete probability tables can be generated for any value of c, m, or n with one of the computer programs provided. When the number of items or raters is large (>25), the right-tail probability associated with any value of a coefficient may be estimated by a z-score procedure. The three coefficients are applicable not only to validity and reliability (test-retest and internal consistency) determinations but also to the item analysis, agreement analysis, and cluster or factor analysis of rating-scale data.

Cited by

• The benefits of being present: Mindfulness and its role in psychological well-being.
• Translation, Adaptation, and Preliminary Validation of the Female Sexual Function Index into Spanish (Colombia)
• Caracterización de los escenarios como herramienta para la planeación pública del agua: El caso mexicano / Characterization of Public Planning Scenarios: The Case of Mexican Water Sector
• Content Validity of a New Soccer (Football) Return-to-Play Test: The RONDO-TEST
• Single-Leg Countermovement Jump Compensation Assessment: Content Validity of a Checklist
• Design, Validation, and Reliability of an Observation Instrument for Technical and Tactical Actions of the Offense Phase in Soccer. [europepmc]
• Comparative Study of Two Intervention Programmes for Teaching Soccer to School-Age Students. [europepmc]
• Translation into modern standard Arabic, cross-cultural adaptation and psychometric properties' evaluation of the Lower Extremity Functional Scale (LEFS) in Arabic-speaking athletes with Anterior Cruciate Ligament (ACL) injury. [europepmc]
• Cross-cultural adaptation and psychometric properties' evaluation of the modern standard Arabic version of Cumberland Ankle Instability Tool (CAIT) in professional athletes. [europepmc]
• The Validity and Reliability of Live Football Match Statistics From Champdas Master Match Analysis System. [europepmc]
• Designing and Validating a Basketball Learning and Performance Assessment Instrument (BALPAI). [europepmc]
• Utilizing Video-Based Trainings to Improve Decision Making in High School Quarterbacks. [europepmc]
• Design and Validation of the Instrument for the Measurement of Learning and Performance in Football. [europepmc]
• Psychometric Properties of the Colombian Version of the HIV Attitudes Scale for Adolescents. [europepmc]
• Cross-Cultural Adaptation and Psychometric Properties of the Spanish Version of the Tampa Scale for Kinesiophobia for Temporomandibular Disorders. [europepmc]
• The Gender Violence - Implicit Association Test to measure attitudes toward intimate partner violence against women. [europepmc]
• COVID-19 anxiety scale (CAS): Development and psychometric properties. [europepmc]
• Attitudes and beliefs of nurses and physicians about managing sexual health in primary care: A multi-site cross-sectional comparative study. [europepmc]
• Development and Psychometric Properties of the Questionnaire for Assessing Educational Podcasts (QAEP). [europepmc]
• Design, Validation, and Reliability of an Observational Instrument for Technical and Tactical Actions in Singles Badminton. [europepmc]
• Validity and Reliability of the Caregiver Strain Index Scale in Women during the Puerperium in Spain. [europepmc]
• A national survey of ambient air pollution health literacy among adult residents of Taiwan. [europepmc]
• The Pedagogical Model of Hybrid Teaching: Difficulties of University Students in the Context of COVID-19. [europepmc]
• Content Validation of a Semi-Structured Interview to Analyze the Management of Suffering. [europepmc]
• Sociodemographic Predictors Associated with the Willingness to Get Vaccinated against COVID-19 in Peru: A Cross-Sectional Survey. [europepmc]
• International physical therapists consensus on clinical descriptors for diagnosing rotator cuff related shoulder pain: A Delphi study. [europepmc]
• Development and psychometric properties of the Stressors in Breast Cancer Scale. [europepmc]
• Development and initial validation of the trait and state Mindful Eating Behaviour Scales. [europepmc]
• Emotion Regulation and Self-Efficacy: The Mediating Role of Emotional Stability and Extraversion in Adolescence. [europepmc]
• The biobehavioural pain and movement questionnaire (BioPMovQ): development and psychometric validation of a new questionnaire. [europepmc]

Related

• Content Validity and Reliability of Single Items or Questionnaires
• Making Reliability Reliable
• Reliability representativeness: How well does coefficient alpha summarize reliability across the score distribution?
• A new approach to reliability assessment based on Exploratory factor analysis
• A Reliability Coefficient for Maximum Likelihood Factor Analysis