A Framework for Conceptualizing and Evaluating the Validity of Instructionally Relevant Assessments

2016/01/02 by James W. Pellegrino, Louis V. DiBello, Susan R. Goldman · 184 citations
Decision Sciences · Psychology · Social Sciences · #Argument (complex analysis) #Cognition #Computer science #Construct (python library) #Construct validity #Curriculum #Developmental psychology #Educational Assessment and Improvement #External validity #Incremental validity #Mathematics education #Pedagogy #Predictive validity #Psychology #Psychometric Methodologies and Testing #Psychometrics #Social psychology #Student Assessment and Feedback #Test validity

paper · doi:10.1080/00461520.2016.1145550

published in Educational Psychologist 51(1), 59-81 (Routledge)

openalex publication_date 2016/01/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Abstract

Assessments that function close to classroom teaching and learning can play a powerful role in fostering academic achievement. Unfortunately, however, relatively little attention has been given to discussion of the design and validation of such assessments. The present article presents a framework for conceptualizing and organizing the multiple components of validity applicable to assessments intended for use in the classroom to support ongoing processes of teaching and learning. The conceptual framework builds on existing validity concepts and focuses attention on three components: cognitive validity, instructional validity, and inferential validity. The goal in presenting the framework is to clarify the concept of validity, including key components of the interpretive argument, while considering the types and forms of evidence needed to construct a validity argument for classroom assessments. The framework's utility is illustrated by presenting an application to the analysis of the validity of assessments embedded within an elementary mathematics curriculum.

Citations

• The Interplay of Evidence and Consequences in the Validation of Performance Assessments

Cited by

• Self-regulated Learning in the Digitally Enhanced Science Classroom: Toward an Early Warning System
• The concept of an agile curriculum as applied to a middle school mathematics digital learning system (DLS)
• Revealing Rubric Relations: Investigating the Interdependence of a Research-Informed and a Machine Learning-Based Rubric in Assessing Student Reasoning in Chemistry
• Validation of an instrument for assessing basic science process skills in initial elementary teacher education
• Validity Arguments for AI‐Based Automated Scores: Essay Scoring as an Illustration
• Helping students to become capable learners
• A Hierarchical Taxonomy of Test Validity for More Flexibility of Validation. [europepmc]
• Observing the World Through Your Own Lenses - The Role of Perceived Adaptability for Epistemological Beliefs About the Development of Scientific Knowledge. [europepmc]
• Validation of Embedded Experience Sampling (EES) for Measuring Non-cognitive Facets of Problem-Solving Competence in Scenario-Based Assessments. [europepmc]
• Technology-related knowledge, skills, and attitudes of pre- and in-service teachers: The current situation and emerging trends. [europepmc]
• Undergraduate Students' Critical Online Reasoning-Process Mining Analysis. [europepmc]
• "Finding the Right Window Into What They're Doing": Assessment of Maker-based Learning Experiences Remotely. [europepmc]

Related

• A General Framework for the Validation of Embedded Formative Assessment
• Validity: on the meaningful interpretation of assessment data
• Current Concerns in Validity Theory
• Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning.
• Potentially engaging mathematics teaching practices in secondary classrooms: Investigating observation tool validity and score reliability