Extrinsic Convergent Validity Evidence to Prevent Jingle and Jangle Fallacies

2020/01/20 by Oscar Gonzalez, Oscar González, David P. MacKinnon +1 · 131 citations
Computer Science · Psychology · #Advanced Statistical Modeling Techniques #Artificial intelligence #Clinical psychology #Cognitive psychology #Computer science #Convergent validity #Grit, Self-Efficacy, and Motivation #Psychology #Psychometrics

paper · open access · doi:10.1080/00273171.2019.1707061

published in Multivariate Behavioral Research 56(1), 3-19 (Taylor & Francis)

openalex publication_date 2020/01/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/25

Abstract

In psychology, there have been vast creative efforts in proposing new constructs and developing measures to assess them. Less effort has been spent in investigating construct overlap to prevent bifurcated literatures, wasted research efforts, and jingle-jangle fallacies. For example, researchers could gather validity evidence to evaluate if two measures with the same label actually assess different constructs (jingle fallacy), or if two measures with different labels actually assess the same construct (jangle fallacy). In this paper, we discuss the concept of extrinsic convergent validity, a source of validity evidence demonstrated when two measures of the same construct, or two measures of seemingly different constructs, have comparable correlations with external criteria. We introduce a formal approach to obtain extrinsic convergent validity evidence using tests of dependent correlations and evaluate the tests using Monte Carlo simulations. Also, we illustrate the methods by examining the overlap between the self-control and grit constructs, and the overlap among seven seemingly different measures of the connectedness to nature construct. Finally, we discuss how extrinsic convergent validity evidence supplements other sources of evidence that support validity arguments of construct overlap.

Citations

• Emotional Affinity toward Nature as a Motivational Basis to Protect Nature
• The Nature Relatedness Scale
• Grit: Perseverance and passion for long-term goals.
• High Self‐Control Predicts Good Adjustment, Less Pathology, Better Grades, and Interpersonal Success
• Much ado about grit: A meta-analytic synthesis of the grit literature.

Cited by

• Construct Diversity in Measures of Helicopter Parenting in Emerging Adulthood
• Untangling graduate employability and work readiness to inform evidence-based practice in higher education
• Descriptive, Predictive and Explanatory Personality Research: Different Goals, Different Approaches, but a Shared Need to Move beyond the Big Few Traits
• The query is the theory: Why “exact-term” bibliometrics can conflate climate anxiety with broader eco-emotions
• The role of personal and contextual factors when investigating technology integration in general and vocational education
• A Systematic Review of Research on Personalized Learning: Personalized by Whom, to What, How, and for What Purpose(s)?
• Emotional Responses to AI Use: Development of the SAG-AI Scale for Shame and Guilt
• Evaluating the Convergence of Dimensions Across Social Evaluation Models
• A systematic review of interpersonal processes and their measurement within experience sampling studies of self-injurious thoughts and behaviours
• Examining the conceptual and empirical distinctiveness of Agreeableness and “dark” personality items
• The measurement of goal dimensions: A critical review.
• Sensory Sensitivity: Implications From the Fields of Medicine and Psychology for a Gifted Population
• Statistical Learning and Language Impairments: Toward More Precise Theoretical Accounts. [europepmc]
• Eudaemonic Well-Being in Midlife Women: Change in and Correspondence Between Concurrent and Retrospective Reports. [europepmc]
• Pathfinder: a gamified measure to integrate general cognitive ability into the biological, medical, and behavioural sciences. [europepmc]
• Expanding the purview of wellness indicators: validating a new measure that includes attitudes, behaviors, and perspectives. [europepmc]
• Interpreting physical sensations to guide health-related behavior : An introductory review on psychosomatic competence. [europepmc]
• Can flipped classroom pedagogy offer promising perspectives for mathematics education on pandemic-related issues? A systematic literature review. [europepmc]
• Using machine-learning strategies to solve psychometric problems. [europepmc]
• Undergraduate Lay Theories of Abilities: Mindset, universality, and brilliance beliefs uniquely predict undergraduate educational outcomes. [europepmc]
• Psychometric validation of four-item exercise identity and healthy-eater identity scales and applications in weight loss maintenance. [europepmc]
• Assessing Discriminant Validity through Structural Equation Modeling: The Case of Eating Compulsivity. [europepmc]
• Designing and evaluating tasks to measure individual differences in experimental psychology: a tutorial. [europepmc]
• Examining the role of personality functioning in a hierarchical taxonomy of psychopathology using two years of ambulatory assessed data. [europepmc]
• Detecting jingle and jangle fallacies by identifying consistencies and variabilities in study specifications - a call for research. [europepmc]
• Physical health mindsets and information avoidance. [europepmc]
• Cognitive, affective and sociological predictors of school performance in mathematics. [europepmc]
• The relationship between climate change and mental health: a systematic review of the association between eco-anxiety, psychological distress, and symptoms of major affective disorders. [europepmc]
• Measures of Subclinical Psychopathy and Everyday Sadism are Still Redundant: A Conceptual Replication and Extension of Blötner and Mokros (2023). [europepmc]
• Addressing the criticisms and critiques of positive psychology: recommendations for improving the science and practice of the field. [europepmc]
• Data from an International Multi-Centre Study of Statistics and Mathematics Anxieties and Related Variables in University Students (the SMARVUS Dataset). [europepmc]
• On the Measurement and Conceptualization of Parental Rules. [europepmc]
• A Measurement Roadmap to Guide Implementation Practitioners in Selecting Implementation Measures. [europepmc]
• A computational validation for the health concept maturity levels questionnaire. [europepmc]
• Construct Diversity in Measures of Helicopter Parenting in Emerging Adulthood. [europepmc]
• Predicting relationship quality with itself? A single general factor captures most of the variance across 34 common relationship measures. [europepmc]
• Imposter syndrome as mediator and moderator between personality and mental health in Malaysian students. [europepmc]

Related

• Construct jangle or construct mangle? Thinking straight about (nonredundant) psychological constructs
• A Validity-Based Framework for Understanding Replication in Psychology
• Causal inference for psychologists who think that causal inference is not for them
• The dire disregard of measurement invariance testing in psychological science.
• Promises and Perils of Experimentation: The Mutual-Internal-Validity Problem