Multilevel Modelling of Country Effects: A Cautionary Tale

2015/05/08 by Mark L. Bryan, Stephen P. Jenkins · 1 voice · 16 citations
Economics, Econometrics and Finance · Social Sciences · #Economic Policies and Impacts #Intergenerational and Educational Inequality Studies #Spatial and Panel Data Analysis

paper · pdf · doi:10.1093/esr/jcv059

openalex publication_date 2015/05/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Abstract

Country effects on outcomes for individuals are often analysed using multilevel (hierarchical) models applied to harmonized multi-country data sets such as ESS, EU-SILC, EVS, ISSP, and SHARE. We point out problems with the assessment of country effects that appear not to be widely appreciated, and develop our arguments using Monte Carlo simulation analysis of multilevel linear and logit models. With large sample sizes of individuals within each country but only a small number of countries, analysts can reliably estimate individual-level effects but estimates of parameters summarizing country effects are likely to be unreliable. Multilevel modelling methods are no panacea.

Citations

• Small Sample Inference for Fixed Effects from Restricted Maximum Likelihood
• A Practitioner’s Guide to Cluster-Robust Inference
• Inference with Difference-in-Differences and Other Panel Data
• Bootstrap-Based Improvements for Inference with Clustered Errors

Cited by

• Firm-stakeholder tension as an antecedent of female board representation in the Middle East and North Africa
• Investigating the Links Between Cultural Values and Belief in Conspiracy Theories: The Key Roles of Collectivism and Masculinity
• Shielding free movement? Reciprocity in welfare institutions and opposition to EU labour immigration
• Career advancement prospects of home-based workers across Europe: a gender and family perspective
• Work group diversity in professional football: An individual‐level approach to the effects of diversity in organizations
• Support for an EU‐wide social policy? A country‐comparative analysis of unemployment perceptions
• Policy Compliance Under Disagreement: The Joint Roles of Duty/Deterrence Beliefs, Symbolic Ideology, and Policy Environment
• Dark and Bright Patterns in Cookie Consent Requests
• Can earned income tax credits earn their keep?: earned income tax credits and in-work poverty in comparative perspective
• Decarbonization Politics for All: Means‐Tested Social Assistance, Eco‐Social Values, and Public Support for Increased Fossil Fuel Taxes in Europe
• Who benefits from the social democratic march to the middle?
• Do EU consumers think about meat reduction when considering to eat a healthy, sustainable diet and to have a role in food system change?
• Impact of perceived corruption on stress: Does social capital serve as an administrative resource?
• Always looking for something better? The impact of job insecurity on turnover intentions: Do employables and irreplaceables react differently?
• The Political Effects of the AI Revolution: Micro-level Evidence
• Where Income Becomes Wealth: How Redistribution Moderates the Association between Income and Wealth

Discussions

• Mark Bryan and I have Stata do-file code for this sort of thing (mimicing EU-SILC data) in the online supplementary material accompanying doi.org/10.1093/esr/... [bsky, 2 points, 0 comments]

Related

• Estimating causal effects of sanctions impacts: what role for country-level studies?
• An Introduction to Multilevel Modeling Techniques
• An inconvenient dataset: bias and inappropriate inference with the multilevel model
• Effect fusion using model-based clustering
• Understanding, Choosing, and Unifying Multilevel and Fixed Effect Approaches