2018/08/31 by Jayson Nissen, Robin A. Donatello, Robin Donatello +4 · 5 citations
Mathematics · Physics and Astronomy · Social Sciences · #FOS: Physical sciences #Physics Education (physics.ed-ph) #Statistical Methods and Bayesian Inference #Survey Methodology and Nonresponse #Survey Sampling and Estimation Techniques #physics.ed-ph
paper · pdf · doi:10.48550/arxiv.1809.00035
openalex publication_date 2018/08/31 · arxiv created 2019/02/19 · arxiv updated 2019/02/20 · openalex created_date 2022/08/03 · openalex updated_date 2026/07/28
Physics education researchers (PER) commonly use complete-case analysis to address missing data. For complete-case analysis, researchers discard all data from any student who is missing any data. Despite its frequent use, no PER article we reviewed that used complete-case analysis provided evidence that the data met the assumption of missing completely at random (MCAR) necessary to ensure accurate results. Not meeting this assumption raises the possibility that prior studies have reported biased results with inflated gains that may obscure differences across courses. To test this possibility, we compared the accuracy of complete-case analysis and multiple imputation (MI) using simulated data. We simulated the data based on prior studies such that students who earned higher grades participated at higher rates, which made the data missing at random (MAR). PER studies seldom use MI, but MI uses all available data, has less stringent assumptions, and is more accurate and more statistically powerful than complete-case analysis. Results indicated that complete-case analysis introduced more bias than MI and this bias was large enough to obscure differences between student populations or between courses. We recommend that the PER community adopt the use of MI for handling missing data to improve the accuracy in research studies.