It Might Not Make a Big DIF
2015/06/29 by R. Philip Chalmers, Alyssa Counsell, David B. Flora · 29 citations
Decision Sciences · Computer Science · Psychology · #Psychometric Methodologies and Testing #Advanced Statistical Modeling Techniques #Cognitive Abilities and Testing
paper · doi:10.1177/0013164415584576
Abstract
Differential test functioning, or DTF, occurs when one or more items in a test demonstrate differential item functioning (DIF) and the aggregate of these effects are witnessed at the test level. In many applications, DTF can be more important than DIF when the overall effects of DIF at the test level can be quantified. However, optimal statistical methodology for detecting and understanding DTF has not been developed. This article proposes improved DTF statistics that properly account for sampling variability in item parameter estimates while avoiding the necessity of predicting provisional latent trait estimates to create two-step approximations. The properties of the DTF statistics were examined with two Monte Carlo simulation studies using dichotomous and polytomous IRT models. The simulation results revealed that the improved DTF statistics obtained optimal and consistent statistical properties, such as obtaining consistent Type I error rates. Next, an empirical analysis demonstrated the application of the proposed methodology. Applied settings where the DTF statistics can be beneficial are suggested and future DTF research areas are proposed.
Citations
Cited by
- Assessing the Robustness of Mixture Models to Measurement Noninvariance
- Model-Based Measures for Detecting and Quantifying Response Bias
- Measuring early childhood development at a global scale: Evidence from the Caregiver-Reported Early Development Instruments
- f MACS : Generalizing d MACS Effect Size for Measurement Noninvariance with Multiple Groups and Multiple Grouping Variables
- Differential item functioning magnitude and impact measures from item response theory models. [europepmc]
- Improving Factor Score Estimation Through the Use of Observed Background Characteristics. [europepmc]
- The Development and Validation of the Bergen-Yale Sex Addiction Scale With a Large National Sample. [europepmc]
- Fitting item response unfolding models to Likert-scale data using mirt in R. [europepmc]
- Plausible-Value Imputation Statistics for Detecting Item Misfit. [europepmc]
- Self-Compassion Scale: IRT Psychometric Analysis, Validation, and Factor Structure - Slovak Translation. [europepmc]
- Assessing the Robustness of Mixture Models to Measurement Noninvariance. [europepmc]
- When Does Differential Item Functioning Matter for Screening? A Method for Empirical Evaluation. [europepmc]
- Measurement Invariance and Psychometric Analysis of Oxford Happiness Inventory Scale across Gender and Marital Status. [europepmc]
- Web-based and mixed-mode cognitive large-scale assessments in higher education: An evaluation of selection bias, measurement bias, and prediction bias. [europepmc]
- Estimating classification consistency of screening measures and quantifying the impact of measurement bias. [europepmc]
- Examining the measurement equivalence of the Maslach Burnout Inventory across age, gender, and specialty groups in US physicians. [europepmc]
- Differential Item Functioning Analyses of the Patient-Reported Outcomes Measurement Information System (PROMIS®) Measures: Methods, Challenges, Advances, and Future Directions. [europepmc]
- Assessing measurement invariance in the EORTC QLQ-C30. [europepmc]
- Practical Assessment of Alcohol Use Disorder in Routine Primary Care: Performance of an Alcohol Symptom Checklist. [europepmc]
- Race and self-reported paranoia: Increased item endorsement on subscales of the SPQ. [europepmc]
- Psychometric Performance of a Substance Use Symptom Checklist to Help Clinicians Assess Substance Use Disorder in Primary Care. [europepmc]
- Psychometric Analysis of the Modified Differential Emotions Scale and the Six-Item Life Orientation Test-Revised in a Cohort of Older Women from the Women's Health Initiative. [europepmc]
- Differential item functioning of material deprivation assessment in households with or without children. [europepmc]
- Ant colony optimization for parallel test assembly. [europepmc]
- Equivalence of Alcohol Use Disorder Symptom Assessments in Routine Clinical Care When Completed Remotely via Online Patient Portals Versus In Clinic via Paper Questionnaires: Psychometric Evaluation. [europepmc]
- Establishing and evaluating the gradient of item naming difficulty in post-stroke aphasia and semantic dementia. [europepmc]
- Understanding inequitable health care: methodological approaches, challenges, and opportunities. [europepmc]
- Measuring visual ability in linguistically diverse populations. [europepmc]
- The Patient Reported Inventory of Self-Management of Chronic Conditions (PRISM-CC): testing for bias across patient characteristics and languages. [europepmc]
Related