Sources of Validity Evidence for Educational and Psychological Tests

2007/11/09 by Gregory J. Cizek, Sharyn L. Rosenberg, Heather H. Koons · 12 citations
Decision Sciences · Computer Science · #Psychometric Methodologies and Testing #Advanced Statistical Modeling Techniques #Reliability and Agreement in Measurement

paper · doi:10.1177/0013164407310130

Abstract

This study investigates aspects of validity reflected in a large and diverse sample of published measures used in educational and psychological testing contexts. The current edition of Mental Measurements Yearbook served as the data source for this study. The validity aspects investigated included perspective on validity represented, number and kinds of sources of validity evidence provided, overall evaluation of the favorability of the test, and whether these factors varied as a function of the type of test. Findings reveal that validity information is not routinely provided in terms of modern validity theory, some sources of validity evidence (e.g., consequential) are essentially ignored in validity reports, and the favorability of judgments about a test is more strongly related to the number of validity sources provided than to the perspective on validity taken or other factors. The article concludes with implications for extending and refining current validity theory and validation practice.

Cited by

• Rethinking the Measurement of Motivational Climate: Understanding and Measuring Student Perceptions in University STEM Classes
• Looking our limitations in the eye: A call for more thorough and honest reporting of study limitations
• A qualitative study of validation practices in classroom-based assessment in health professions education: what do faculty members actually do?
• Characterizing the literature on validity and assessment in medical education: a bibliometric study. [europepmc]
• Fostering implementation of knowledge into health practice: study protocol for the validation and redevelopment of the Knowledge Uptake and Utilization Tool. [europepmc]
• Assessing Distinctiveness in Multidimensional Instruments Without Access to Raw Data - A Manifest Fornell-Larcker Criterion. [europepmc]
• Questionnaire validation practice within a theoretical framework: a systematic descriptive literature review of health literacy assessments. [europepmc]
• COVID-19 as the tipping point for integrating e-assessment in higher education practices. [europepmc]
• Construct validity of international literacy measures: implications for dyslexia across cultures. [europepmc]
• Psychometric Properties of Controlled Oral Word Association (COWA) Test and Associations With Education and Bilingualism in American Indian Adults: The Strong Heart Study. [europepmc]
• Psychometric reliability, validity, and generalizability of 3MSE scores among American Indian adults: the Strong Heart Study. [europepmc]
• A Scoping Review of Assessments in Undergraduate Medical Education: Implications for Residency Programs and Medical Schools. [europepmc]

Related

• Current Concerns in Validity Theory
• The Meaning of Validity in the New Standards for Educational and Psychological Testing:
• Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning.
• Validity: on the meaningful interpretation of assessment data
• The Concept of Validity.