Total Survey Error: Past, Present, and Future
2010/01/01 by Robert M. Groves, R. M. Groves, L. Lyberg +1 · 598 citations
Economics, Econometrics and Finance · Engineering · Mathematics · Psychology · Social Sciences · #Computer science #Data science #Engineering #Focus (optics) #Healthcare Policy and Management #Management science #Mathematics #Population #Psychology #Sample (material) #Statistical Methods and Bayesian Inference #Statistics #Survey Methodology and Nonresponse #Survey data collection #Survey methodology #Survey research #Survey sampling
paper · doi:10.1093/poq/nfq065
published in Public Opinion Quarterly 74(5), 849-879 (Oxford University Press)
openalex publication_date 2010/01/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06
Abstract
“Total survey error” is a conceptual framework describing statistical error properties of sample survey statistics. Early in the history of sample surveys, it arose as a tool to focus on implications of various gaps between the conditions under which probability samples yielded unbiased estimates of finite population parameters and practical situations in implementing survey design. While the framework permits design-based estimates of various error components, many of the design burdens to produce those estimates are large, and in practice most surveys do not implement them. Further, the framework does not incorporate other, nonstatistical, dimensions of quality that are commonly utilized in evaluating statistical information. The importation of new modeling tools brings new promise to measuring total survey error components, but also new challenges. A lasting value of the total survey error framework is at the design stage of a survey, to attempt a balance of costs and various errors. Indeed, this framework is the central organizing structure of the field of survey methodology.
Cited by
- Welcher Zweck heiligt die Mittel? Bemerkungen zur Repräsentativitätsdebatte in der Meinungsforschung
- Interviewer Training Guidelines of Multinational Survey Programs: A Total Survey Error Perspective
- Digital Data, Administrative Data, and Survey Compared: Updating the Classical Toolbox for Assessing Data Quality of Big Data, Exemplified by the Generation of Corruption Data
- The State’s Politics of “Fake Data”
- Conditioning Public Opinion Perceptions by “Survey Methods 101”: Informing, Engaging, and Motivating Individuals for Critical Processing of Public Opinion Polls
- Total Survey Error in Biomeasure Data Collection
- Nonresponse versus Measurement Error
- When Synthetic Users Fail: A Cross-Domain Benchmark of LLM-Simulated Human Survey Responses
- Total Bias in Income Surveys when Nonresponse and Measurement Errors are Correlated
- Evaluation of Biases in Self-reported Demographic and Psychometric Information: Traditional versus Facebook-based Surveys
- Streamlining business functions in official statistical production with Machine Learning
- Causal Inferences from Digital Behavioral Data
- Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models
- Interviewer Involvement in Respondent Selection Moderates the Relationship Between Response Rates and Sample Bias in Cross-National Survey Projects in Europe
- Integrating smart surveys with traditional surveys
- Two implications of survey research mode during war: evidence from Russia’s full-scale invasion of Ukraine
- Methods, Developments, and Technological Innovations for Population Surveys
- The Long-Term Impact of Different Offline Population Inclusion Strategies in Probability-Based Online Panels: Evidence From the German Internet Panel and the GESIS Panel
- Little mentioning, moderate attention, great relevance: The quality of online platform data in the Digital Services Act
- From ideal experiments to ideal research designs (IDRs): What they are and why we should use them more
- Thinking Clearly About Sampling and Representation With the Total Survey Error Framework
- Emulating Public Opinion: A Proof-of-Concept of AI-Generated Synthetic Survey Responses for the Chilean Case
- Weights and Methodology Brief for the COVID-19 Symptom Survey by University of Maryland and Carnegie Mellon University, in Partnership with Facebook
- Online Surveys are Mixed-Device Surveys. Issues Associated with the Use of Different (Mobile) Devices in Web Surveys
- Polling bias and undecided voter allocations: US Presidential elections, 2004 - 2016
- Digital trace data collection through data donation
- Determinants of participation in a longitudinal survey during the COVID-19 pandemic: the case of a low-infection country
- Relative Bias in Health Estimates from Probability-Based Online Panels: Systematic Review and Meta-Analysis
- A Total Statistical Error Framework for Comparing Census Data Collection Methods
- A Sensitivity Analysis Framework for Quantifying Confidence in Decisions in the Presence of Data Uncertainty
- Health Care Provider Surveys in the United States, 2000–2010
- Survey sampling [wikipedia]
- Managing an online survey about influenza vaccination in primary healthcare workers. [europepmc]
- Mode differences in a mixed-mode health interview survey among adults. [europepmc]
- Estimate of adolescent alcohol use in China: a meta-analysis. [europepmc]
- Estimate of undergraduate university student alcohol use in China: a systematic review and meta-analysis. [europepmc]
- Mixing modes in a population-based interview survey: comparison of a sequential and a concurrent mixed-mode design for public health research. [europepmc]
- Mode Equivalence of Health Indicators Between Data Collection Modes and Mixed-Mode Survey Designs in Population-Based Health Interview Surveys for Children and Adolescents: Methodological Study. [europepmc]
- Principles to Guide National Data Collection on the Health of Persons in the Criminal Justice System. [europepmc]
- Cognitive and Affective Processing of Risk Information: A Survey Experiment on Risk-Based Decision-Making Related to Crime and Public Safety. [europepmc]
- Addressing Unintentional Exclusion of Vulnerable and Mobile Households in Traditional Surveys in Kathmandu, Dhaka, and Hanoi: a Mixed-Methods Feasibility Study. [europepmc]
- The Quality of Data on Participation in Adult Education and Training. An Analysis of Varying Participation Rates and Patterns Under Consideration of Survey Design and Measurement Effects. [europepmc]
- A global dataset of inland fisheries expert knowledge. [europepmc]
- The interplay of migration and cultural similarity between countries: Evidence from Facebook data on food and drink interests. [europepmc]
- A systematic literature review of how and whether social media data can complement traditional survey data to study public opinion. [europepmc]
- Using Smartphone Sensor Paradata and Personalized Machine Learning Models to Infer Participants' Well-being: Ecological Momentary Assessment. [europepmc]
- Survey mode and nonresponse bias: A meta-analysis based on the data from the international social survey programme waves 1996-2018 and the European social survey rounds 1 to 9. [europepmc]
- When survey science met web tracking: Presenting an error framework for metered data. [europepmc]
- Comparing national device-based physical activity surveillance systems: a systematic review. [europepmc]
- Owning, Renting and Environmental Proactivity: The Role of Housing Tenure in Hypothetical Housing Decisions. [europepmc]
- Evaluating the accuracy of survey data: a case study of COVID-19 vaccination rates in Germany. [europepmc]
- Conditioning Public Opinion Perceptions by "Survey Methods 101": Informing, Engaging, and Motivating Individuals for Critical Processing of Public Opinion Polls. [europepmc]
- Computer-Assisted Mobile Phone Interviews in Low- and Middle-Income Countries Through a Total Survey Error Framework. [europepmc]
- Big data, big problems: Responding to "Are we there yet?" [europepmc]
- Exploring the big data paradox for various estimands using vaccination data from the global COVID-19 Trends and Impact Survey (CTIS) [europepmc]
Related