Specification curve analysis

2020/07/27 by Uri Simonsohn, Joseph P. Simmons, Leif D. Nelson · 84 citations
Social Sciences · Computer Science · #Names, Identity, and Discrimination Research #Computational and Text Analysis Methods #Authorship Attribution and Profiling

paper · doi:10.1038/s41562-020-0912-z

Cited by

• <i>Instrumentally</i> Inclusive: The Political Psychology of Homonationalism
• Robustness is better assessed with a few thoughtful models than with billions of regressions
• Reducing bias, increasing transparency and calibrating confidence with preregistration
• Possible futures all at once: time frame and time lag in short-term longitudinal media effects research on well-being
• Nonreplicable publications are cited more than replicable ones
• Bestiary of Questionable Research Practices in Psychology
• Learning from Data: An Empirics-First Approach to Relevant Knowledge Generation
• Are our 95% CIs only worth 45% confidence? A heterogeneity-aware method for extending inference
• Do Citizens Stereotype Muslims as an Illiberal <i>Bogeyman</i>? Evidence from a Double-List Experiment
• Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology
• Milliways: Taming Multiverses through Principled Evaluation of Data Analysis Paths
• Post-selection Inference in Multiverse Analysis (PIMA): An Inferential Framework Based on the Sign Flipping Score Test
• Respiratory and cardiac interoceptive sensitivity in the first two years of life
• E-Values For Multiplicity Control In Multiverse Analysis
• PlotTwist: A Creative Plot Generation Framework with Small Language Models
• A multiverse-consensus pipeline for reproducible feature selection in untargeted LC-MS metabolomics
• Auditing Differential Visibility of Political Content on TikTok
• Mitigating LLM-based p-Hacking by Preregistering for the Next LLM
• Immigrant Residential Segregation in Europe: A Comparative Study of Spatial Segregation Patterns in Urban Areas across 30 Countries
• The threat of analytic flexibility in using large language models to simulate human data
• Privacy calculus, privacy paradox, and context collapse: A replication of three key studies in communication privacy research
• Grilling the data: application of specification curve analysis to red meat and all-cause mortality
• Birth‐order effects on risk taking are limited to the family environment
• Many AI Analysts, One Dataset: Navigating the Agentic Data Science Multiverse
• Enhancing precision in human neuroscience
• Long-term effects of the Targeting the Ultra Poor program—A reproducibility and replicability assessment of Banerjee et al. (2021)
• Does alleviating poverty increase cognitive performance? Short- and long-term evidence from a randomized controlled trial
• Do decisions about outliers and influential effects matter? Evidence from 358 behavioral science meta-analyses
• Generalizability of choice architecture interventions
• Exploring the Garden of Forking Paths in Empirical Software Engineering Research: A Multiverse Analysis
• Single-Dataset Meta-Analysis For Many-Analysts And Multiverse Studies
• Respiratory and Cardiac Interoceptive Sensitivity in the First Two Years of Life
• Investigating the analytical robustness of the social and behavioural sciences
• Addressing researcher degrees of freedom through minP adjustment
• Towards a Geometric Characterization of Multiverse Analysis
• Need for Cognition is associated with a preference for higher task load in effort discounting
• Reproducibility and robustness of economics and political science research
• A Causal Inference Framework for Climate Change Attribution in Ecology
• No clear evidence for a domain-general violation of expectation effect in the pupillary responses of 9- to 10-month-olds
• Message quality and audience characteristics shape evaluation and impact of real-world social media climate communication
• No meta-analytical effect of economic inequality on well-being or mental health
• Attention-Deficit/Hyperactivity Disorder Symptoms as a Function of Arousal and Executive Vigilance: Testing Halperin and Schulz’s Neurodevelopmental Model in a Sample of Community Adults
• On the reliability of value-modulated attentional capture: An online replication and multiverse analysis
• Are there limits to empathy? A survey experiment on empathic concern and perspective-taking as bases for attitudes towards different groups of refugees
• Assessing the balance between excitation and inhibition in chronic pain through the aperiodic component of EEG
• The Agentic Garden of Forking Paths
• Momentary stress and social context among adolescents at risk for suicide: An experience sampling study
• Learning loss due to school closures during the COVID-19 pandemic
• Exploring associations between affect and marijuana use in everyday life via specification curve analysis.
• Cortical thickness changes precede high levels of amyloid by at least seven years
• A protocol for structured robustness reproductions and replicability assessments
• Predicting replication outcomes and generalizability of results
• Toward a more credible assessment of the credibility of science by many-analyst studies
• Open science, the replication crisis, and environmental public health
• Does the Method Affect the Outcome? How Measures of Partisan Slant of Media Outlets and Affective Polarization Drive Results About Polarization in the United States
• Political segregation in the US workplace
• Will AI ruin the social sciences — or revolutionize them?
• Instability of Global Burden of Disease Estimates of Deaths and DALYs from Major Risk Factors
• Estimating the replicability of Brazilian biomedical science
• Neural Indicators Of Food Cue Reactivity, Regulation, And Valuation And Their Associations With Body Composition And Daily Eating Behavior. [europepmc]
• Nonreplicable publications are cited more than replicable ones. [europepmc]
• Leveraging vibration of effects analysis for robust discovery in observational biomedical data science. [europepmc]
• Toward more rigorous and informative nutritional epidemiology: The rational space between dismissal and defense of the status quo. [europepmc]
• Consensus-based guidance for conducting and reporting multi-analyst studies. [europepmc]
• Multimethod assessment of pubertal timing and associations with internalizing psychopathology in early adolescent girls. [europepmc]
• Coping with COVID: risk and resilience factors for mental health in a German representative panel study. [europepmc]
• Age-related change in task-evoked amygdala-prefrontal circuitry: A multiverse approach with an accelerated longitudinal cohort aged 4-22 years. [europepmc]
• Hidden talents in context: Cognitive performance with abstract versus ecological stimuli among adversity-exposed youth. [europepmc]
• ASAS-NANP symposium: mathematical modeling in animal nutrition: limitations and potential next steps for modeling and modelers in the animal sciences. [europepmc]
• Of differing methods, disputed estimates and discordant interpretations: the meta-analytical multiverse of brain volume and IQ associations. [europepmc]
• Open and reproducible neuroimaging: From study inception to publication. [europepmc]
• In COVID-19 Health Messaging, Loss Framing Increases Anxiety with Little-to-No Concomitant Benefits: Experimental Evidence from 84 Countries. [europepmc]
• How well do covariates perform when adjusting for sampling bias in online COVID-19 research? Insights from multiverse analyses. [europepmc]
• Sensitive periods in development and risk for psychiatric disorders and related endpoints: a systematic review of child maltreatment findings. [europepmc]
• Hop to it! The impact of organization type on innovation response time to the COVID-19 crisis. [europepmc]
• Exploring associations between affect and marijuana use in everyday life via specification curve analysis. [europepmc]
• Beyond Screen Time: The Different Longitudinal Relations between Adolescents' Smartphone Use Content and Their Mental Health. [europepmc]
• Enhancing precision in human neuroscience. [europepmc]
• Need for Cognition is associated with a preference for higher task load in effort discounting. [europepmc]
• Rhythmic visual stimulation as a window into early brain development: A systematic review. [europepmc]
• On the reliability of value-modulated attentional capture: An online replication and multiverse analysis. [europepmc]
• Brain change trajectories in healthy adults correlate with Alzheimer's related genetic variation and memory decline across life. [europepmc]
• Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology. [europepmc]
• Assessing the predictive value of peak alpha frequency for the sensitivity to pain. [europepmc]