It Might Not Make a Big DIF
2015/06/29 by R. Philip Chalmers, Alyssa Counsell, David B. Flora · 132 citations
Computer Science · Decision Sciences · Mathematics · Psychology · #Advanced Statistical Modeling Techniques #Artificial intelligence #Cognitive Abilities and Testing #Computer science #Data mining #Differential item functioning #Econometrics #Item response theory #Machine learning #Mathematics #Monte Carlo method #Polytomous Rasch model #Psychometric Methodologies and Testing #Psychometrics #Statistical hypothesis testing #Statistics #Type I and type II errors
paper · open access · doi:10.1177/0013164415584576
published in Educational and Psychological Measurement 76(1), 114-140 (SAGE Publishing)
openalex publication_date 2015/06/29 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/25
Abstract
Differential test functioning, or DTF, occurs when one or more items in a test demonstrate differential item functioning (DIF) and the aggregate of these effects are witnessed at the test level. In many applications, DTF can be more important than DIF when the overall effects of DIF at the test level can be quantified. However, optimal statistical methodology for detecting and understanding DTF has not been developed. This article proposes improved DTF statistics that properly account for sampling variability in item parameter estimates while avoiding the necessity of predicting provisional latent trait estimates to create two-step approximations. The properties of the DTF statistics were examined with two Monte Carlo simulation studies using dichotomous and polytomous IRT models. The simulation results revealed that the improved DTF statistics obtained optimal and consistent statistical properties, such as obtaining consistent Type I error rates. Next, an empirical analysis demonstrated the application of the proposed methodology. Applied settings where the DTF statistics can be beneficial are suggested and future DTF research areas are proposed.
Citations
Cited by
- Assessing the Robustness of Mixture Models to Measurement Noninvariance
- Model-Based Measures for Detecting and Quantifying Response Bias
- Measuring early childhood development at a global scale: Evidence from the Caregiver-Reported Early Development Instruments
- f MACS : Generalizing d MACS Effect Size for Measurement Noninvariance with Multiple Groups and Multiple Grouping Variables
- Enhancing measurement validity in diverse populations: Modern approaches to evaluating differential item functioning
- Remote Testing of Reading Comprehension in 8-Year-Old Children: Mode and Setting Effects
- Differential item functioning magnitude and impact measures from item response theory models. [europepmc]
- Improving Factor Score Estimation Through the Use of Observed Background Characteristics. [europepmc]
- The Development and Validation of the Bergen-Yale Sex Addiction Scale With a Large National Sample. [europepmc]
- Fitting item response unfolding models to Likert-scale data using mirt in R. [europepmc]
- Plausible-Value Imputation Statistics for Detecting Item Misfit. [europepmc]
- Self-Compassion Scale: IRT Psychometric Analysis, Validation, and Factor Structure - Slovak Translation. [europepmc]
- Assessing the Robustness of Mixture Models to Measurement Noninvariance. [europepmc]
- When Does Differential Item Functioning Matter for Screening? A Method for Empirical Evaluation. [europepmc]
- Measurement Invariance and Psychometric Analysis of Oxford Happiness Inventory Scale across Gender and Marital Status. [europepmc]
- Web-based and mixed-mode cognitive large-scale assessments in higher education: An evaluation of selection bias, measurement bias, and prediction bias. [europepmc]
- Estimating classification consistency of screening measures and quantifying the impact of measurement bias. [europepmc]
- Examining the measurement equivalence of the Maslach Burnout Inventory across age, gender, and specialty groups in US physicians. [europepmc]
- Differential Item Functioning Analyses of the Patient-Reported Outcomes Measurement Information System (PROMIS®) Measures: Methods, Challenges, Advances, and Future Directions. [europepmc]
- Assessing measurement invariance in the EORTC QLQ-C30. [europepmc]
- Practical Assessment of Alcohol Use Disorder in Routine Primary Care: Performance of an Alcohol Symptom Checklist. [europepmc]
- Race and self-reported paranoia: Increased item endorsement on subscales of the SPQ. [europepmc]
- Psychometric Performance of a Substance Use Symptom Checklist to Help Clinicians Assess Substance Use Disorder in Primary Care. [europepmc]
- Psychometric Analysis of the Modified Differential Emotions Scale and the Six-Item Life Orientation Test-Revised in a Cohort of Older Women from the Women's Health Initiative. [europepmc]
- Differential item functioning of material deprivation assessment in households with or without children. [europepmc]
- Ant colony optimization for parallel test assembly. [europepmc]
- Equivalence of Alcohol Use Disorder Symptom Assessments in Routine Clinical Care When Completed Remotely via Online Patient Portals Versus In Clinic via Paper Questionnaires: Psychometric Evaluation. [europepmc]
- Establishing and evaluating the gradient of item naming difficulty in post-stroke aphasia and semantic dementia. [europepmc]
- Understanding inequitable health care: methodological approaches, challenges, and opportunities. [europepmc]
- Measuring visual ability in linguistically diverse populations. [europepmc]
- The Patient Reported Inventory of Self-Management of Chronic Conditions (PRISM-CC): testing for bias across patient characteristics and languages. [europepmc]
Related