Detecting Misspecified Multilevel Structural Equation Models with Common Fit Indices: A Monte Carlo Study

2015/03/04 by Hsien‐Yuan Hsu, Hsien-Yuan Hsu, Oi-man Kwok +4 · 122 citations
Computer Science · Decision Sciences · Mathematics · #Advanced Statistical Modeling Techniques #Covariance #Econometrics #Factor analysis #Mathematics #Monte Carlo method #Psychometric Methodologies and Testing #Sample size determination #Statistics #Structural equation modeling #Technology Adoption and User Behaviour

paper · doi:10.1080/00273171.2014.977429

published in Multivariate Behavioral Research 50(2), 197-215 (Taylor & Francis)

openalex publication_date 2015/03/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04

Abstract

This study investigated the sensitivity of common fit indices (i.e., RMSEA, CFI, TLI, SRMR-W, and SRMR-B) for detecting misspecified multilevel SEMs. The design factors for the Monte Carlo study were numbers of groups in between-group models (100, 150, and 300), group size (10, 20, 30, and 60), intra-class correlation (low, medium, and high), and the types of model misspecification (Simple and Complex). The simulation results showed that CFI, TLI, and RMSEA could only identify the misspecification in the within-group model. Additionally, CFI, TLI, and RMSEA were more sensitive to misspecification in pattern coefficients while SRMR-W was more sensitive to misspecification in factor covariance. Moreover, TLI outperformed both CFI and RMSEA in terms of the hit rates of detecting the within-group misspecification in factor covariance. On the other hand, SRMR-B was the only fit index sensitive to misspecification in the between-group model and more sensitive to misspecification in factor covariance than misspecification in pattern coefficients. Finally, we found that the influence of ICC on the performance of targeted fit indices was trivial.

Cited by

• The performance of Bayesian fit measures in detecting misspecified multilevel structural equation modeling
• Adoption of TikTok as a Learning Tool in Physical Education: Evidence from the Philippines
• The Structure of Informal Learning in the Workplace—An Experience Sampling Approach
• Student assessment of teaching as a source of information about aspects of teaching quality in multiple subject domains: an application of multilevel bifactor structural equation modeling. [europepmc]
• A Solution to Modeling Multilevel Confirmatory Factor Analysis with Data Obtained from Complex Survey Sampling to Avoid Conflated Parameter Estimates. [europepmc]
• What Is the Structure of Time? A Study on Time Perspective in the United States, Poland, and Nigeria. [europepmc]
• Cross-Level Group Measurement Invariance When Groups Are at Different Levels of Multilevel Data. [europepmc]
• Not getting high with a little help from your friends: Social versus drug network correlates of marijuana use among YMSM. [europepmc]
• Model Fit Estimation for Multilevel Structural Equation Models. [europepmc]
• Examining the Impact of and Sensitivity of Fit Indices to Omitting Covariates Interaction Effect in Multilevel Multiple-Indicator Multiple-Cause Models. [europepmc]
• Integration of discrete and global structures of affect across three large samples: Specific emotions within-persons and global affect between-persons. [europepmc]
• Trickle-Down Effects of Entrepreneurial Bricolage and Business Model Innovation on Employee Creativity: Evidence From Entrepreneurial Internet Firms in China. [europepmc]
• Testing the Performance of Level-Specific Fit Evaluation in MCFA Models With Different Factor Structures Across Levels. [europepmc]
• Determination of factors influencing young adults' intention to have COVID-19 vaccine in the Philippines: An integration of Health Belief Model and the Theory of Planned Behavior. [europepmc]
• Estimating and Testing Random Intercept Multilevel Structural Equation Models with Model Implied Instrumental Variables. [europepmc]
• Evaluating fit indices in a multilevel latent growth model with unbalanced design: a Monte Carlo study. [europepmc]
• Self-Compassion and Emotion Dysregulation Mediate the Effect of Stress Appraisals on Elder Abuse and Neglect Behaviors in Dementia Family Caregiving. [europepmc]
• Deterministic processes influence bacterial more than fungal community assembly during the development of biological soil crusts in the desert ecosystem. [europepmc]
• Understanding the Within- and Between-Person Structure of Daily Psychopathology Among Adolescents and Young Adults. [europepmc]
• Supporting implementation of universal prevention initiatives in K-12 schools: impacts on fidelity through organizational readiness and team functioning in a cluster-randomized trial. [europepmc]
• Cross-Lagged Relationships Between Cognitive Ability and Math Achievement. [europepmc]
• Contribution of Emotion Dynamics to Adolescent Psychosocial Well-Being: Protocol for a Longitudinal Study. [europepmc]
• When our measurements are different every day: an ML-SEM simulation study on within-person nonuniform measurement bias in intensive longitudinal data. [europepmc]
• The impact of college students' social anxiety on human-machine trust: the mediating role of perceived intelligence and the moderating role of psychological anthropomorphism. [europepmc]

Related

• The Effect of Estimation Methods on SEM Fit Indices
• Understanding the Model Size Effect on SEM Fit Indices
• Assessing the Size of Model Misfit in Structural Equation Models
• How robust are Structural Equation Models to model miss-specification? A simulation study
• Understanding the Comparative Fit Index: It's all about the base!