Equivalence Testing for Psychological Research: A Tutorial
2018/06/01 by Daniël Lakens, Anne M. Scheel, Peder Mortvedt Isager · 1 voice · 123 citations
Psychology · Computer Science · #Mental Health Research Topics #Advanced Statistical Modeling Techniques
paper · doi:10.1177/2515245918770963
Abstract
Psychologists must be able to test both for the presence of an effect and for the absence of an effect. In addition to testing against zero, researchers can use the two one-sided tests (TOST) procedure to test for equivalence and reject the presence of a smallest effect size of interest (SESOI). The TOST procedure can be used to determine if an observed effect is surprisingly small, given that a true effect at least as extreme as the SESOI exists. We explain a range of approaches to determine the SESOI in psychological science and provide detailed examples of how equivalence tests should be performed and reported. Equivalence tests are an important extension of the statistical tools psychologists currently use and enable researchers to falsify predictions about the presence, and declare the absence, of meaningful effects.
Cited by
- The Practical Alternative to the p Value Is the Correctly Used p Value
- Exploratory hypothesis tests can be more compelling than confirmatory hypothesis tests
- Iconic versus working memory metacognition to evaluate the richness of perception: a registered report
- AffectTracker: real-time continuous rating of affective experience in immersive virtual reality
- Advancing the psychology of social class with large-scale replications in four countries
- A tutorial for calculating field-specific effect size distributions
- Inferential Statistics as Descriptive Statistics: There Is No Replication Crisis if We Don’t Expect Replication
- Nonsignificance misinterpreted as an effect’s absence in psychology: prevalence and temporal analyses
- Positive Images of Heat Waves Reduce the Effects of Messages About Extreme Heat among U.S. Adults
- The equivalence of psychodynamic therapy and cognitive behavioral therapy for depressive disorders in adults: A meta‐analytic review
- Using anchor-based methods to determine the smallest effect size of interest
- The (in)effectiveness of populist rhetoric: a conjoint experiment of campaign messaging
- An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports
- Accurately Identifying Language Disorder in School-Age Children Using Dynamic Assessment of Narrative Language
- Respiratory and cardiac interoceptive sensitivity in the first two years of life
- Validating genuine changes in heartbeat-evoked potentials using pseudotrials and surrogate procedures
- VesselBoost: A Python Toolbox for Small Blood Vessel Segmentation in Human Magnetic Resonance Angiography Data
- Retail Trader's Ruin: An Anatomy of Popular Signal Failure
- Active Listening in Integrative Negotiation
- One Decade Into the Replication Crisis, How Have Psychological Results Changed?
- Defending democracy: investigating the efficacy of elite democratic defence in a competitive information environment
- From means to meaning in the study of sex/gender differences and similarities
- Advancements of phonetics in the 21st century: Quantitative data analysis
- Hunger shifts attention and attribute weighting in dietary choice
- Replicability and transparency in physical therapy research: Time to wake up
- A Guide for Calculating Study-Level Statistical Power for Meta-Analyses
- Warning people that they are being microtargeted fails to eliminate persuasive advantage
- Biased AI writing assistants shift users’ attitudes on societal issues
- Impact of Exposure to Health Misinformation on Belief in Health Misinformation: A Meta-Analysis of RCTs
- Feedback control of a cupola - concepts and experimental results
- Evidence for the irrelevance of irrelevant events
- Beyond Conversations: Spatially-Anchored Previews for Intent Disambiguation in LLM-Assisted Geometry Editing in Virtual Reality
- Parental socioeconomic status weakly predicts specific cognitive and academic skills beyond general cognitive ability
- What about the Male Victims? Exploring the Impact of Gender Stereotyping on Implicit Attitudes and Behavioural Intentions Associated with Intimate Partner Violence
- Civil disobedience in scientific authorship: Resistance and insubordination in science
- The Relationship Between Team Diversity and Team Performance: Reconciling Promise and Reality Through a Comprehensive Meta-Analysis Registered Report
- Respiratory and Cardiac Interoceptive Sensitivity in the First Two Years of Life
- From significance testing to estimation and Open Science: How esci can help
- Meaningful Associations Redux: Quantifying and interpreting effect size in the context of the Adolescent Brain and Cognitive Development study
- Three-Sided Testing to Establish Practical Significance: A Tutorial
- Bridging Null Hypothesis Testing and Estimation: A Practical Guide to Statistical Conclusion Drawing From Research in Psychology
- A gentle introduction to Bayesian statistics, with applications to bilingualism research
- Same model, same data, but different outcomes: Evaluating the impact of method choices in structural equation modeling
- Investigating the reversed congruency effect of gaze through the lens of joint attention
- Diverse platforms, diverse effects: Evidence from a 100-day study on social media and adolescent mental health
- How (not) to demonstrate unconscious priming: Overcoming issues with post-hoc data selection, low power, and frequentist statistics
- Beyond dimension reduction: Stable electric fields emerge from and allow representational drift
- Novelty, Consistency, Transparency: The Trilemma of Psychological Sciences and its Consequences on Open Science Practices
- Learning with multiple external representations in physics: Concreteness fading versus simultaneous presentation
- Can narrative help people engage with and understand information without being persuasive? An empirical study
- Do They Look the Same Unless They Are Angry? Investigating the Other-Race Effect in the Presence of Angry Expressions
- GradInf: Gradient Estimation as Probabilistic Inference
- Is There a Foreign Language Effect on Workplace Bribery Susceptibility? Evidence from a Randomized Controlled Vignette Experiment
- Implicit and explicit attitudes toward people with physical disabilities among clinicians, rehabilitation assistants, and other occupations: A comparative study
- Protecting against researcher bias in secondary data analysis: challenges and potential solutions
- Signaling (in)tolerance: Social evaluation and metaethical relativism and objectivism
- When Algorithms Err : Differential Impact of Early vs. Late Errors on Users’ Reliance on Algorithms
- Simulation-Based Power Analysis for Factorial Analysis of Variance Designs
- No meta-analytical effect of economic inequality on well-being or mental health
- The design and analysis of marine multiple driver experiments
- The replication crisis has led to positive structural, procedural, and community changes
- Common methodological mistakes
- Putting the Spotlight Back Onto the Flanker Task in Autism: Autistic Adults Show Increased Interference from Foils Compared with Non-autistic Adults
- The use and interpretation of unstandardized and standardized effect sizes in psychology: current practices and challenges
- Egg Traders Do Not Discriminate Between Familiar and Unfamiliar Partners
- Lonely Individuals Show Distinct Patterns of Social Media Engagement
- Practical Relevance: A Formal Definition
- Towards better hypothesis tests in oxytocin research: Evaluating the validity of auxiliary assumptions
- How Many Participants Do I Need to Test an Interaction? Conducting an Appropriate Power Analysis and Achieving Sufficient Power to Detect an Interaction
- The cognitive and neural bases of creative thought: A cross-domain systematic review and meta-analysis of transcranial direct current stimulation studies
- Social media use in adolescents with and without mental health conditions
- The Effect of Concrete Wording on Truth Judgements: A Preregistered Replication and Extension of Hansen & Wänke ()
- Don’t Ditch the Laptop Just Yet: A Direct Replication of Mueller and Oppenheimer’s (2014) Study 1 Plus Mini Meta-Analyses Across Similar Studies
- Effects of unconditional cash transfers on family processes and wellbeing among mothers with low incomes
- Reliability Underreported in Verbal Memory Tests for Spanish-Speaking Populations: A Systematic Review
- Does the mere presence of smartphones impact cognition in the high school classroom?
- Vulnerability Appeals in the COVID-19 Pandemic: Insights from a National Survey Experiment
- Tempered Expectations: A Tutorial for Calculating and Interpreting Prediction Intervals in the Context of Replications
- Subjective orgasm experience in different-sex and same-sex couples: A dyadic approach
- Evaluating Effect Size in Psychological Research: Sense and Nonsense
- Ten simple rules for writing a Registered Report
- From arbitrary metrics to meaningful change: Empirical smallest effect sizes of interest for the Feeling Scale during exercise
- Heartbeat-evoked responses in M/EEG: A systematic review of methods with suggestions for analysis and reporting
- Teenagers, screens and social media: a narrative review of reviews and key studies
- Bio-banding in academy football: player’s perceptions of a maturity matched tournament
- Validating Generative Agent-Based Models for Logistics and Supply Chain Management Research
- Virtual-reality induced nostalgia: Duration and psychological benefits
- Will Human-Animal Chimeras Cause Moral Confusion? Exploring Public Attitudes
- From theory to research: Interpretational guidelines, statistical guidance, and a shiny app for the model of excellencism and perfectionism
- Metrological uncertainty and the value of informative nulls in marketing research
- Bridging Cultures in the Era of Big Data: A Cross-Language Equivalence Framework in Machine-Learning Research With Social Media Texts
- Sight vs. Sound Judgments of Music Performance Depend on Relative Performer Quality: Cross-cultural Evidence From Classical Piano and Tsugaru Shamisen Competitions
- Working memory capacity is related to eyewitness identification accuracy, but selective attention and need for cognition are not
- Designing Effective Digital Literacy Interventions for Boosting Deepfake Discernment
- Continuous measurement of pulse arrival time for identification of blood pressure changes in patients undergoing hemodialysis—a feasibility study
- A non-inferiority test for R-squared with random regressors
- Concerns About Replicability Across Two Crises in Social Psychology
- Maybe a free thinker but not a critical one: High conspiracy belief is associated with low critical thinking ability
- Early and later susceptibility to effects of parenting: Exploring Frankenhuis and Panchanathan’s (2011) evolutionary commitment hypothesis.
- Estimation for Better Inference in Neuroscience. [europepmc]
- How Many Participants Do We Have to Include in Properly Powered Experiments? A Tutorial of Power Analysis with Reference Tables. [europepmc]
- Comparing Traditional and Digitized Cognitive Tests Used in Standard Clinical Evaluation - A Study of the Digital Application Minnemera. [europepmc]
- Validation of Highly Accelerated Wave-CAIPI SWI Compared with Conventional SWI and T2*-Weighted Gradient Recalled-Echo for Routine Clinical Brain MRI at 3T. [europepmc]
- Indices of Effect Existence and Significance in the Bayesian Framework. [europepmc]
- Risk communication in tables versus text: a registered report randomized trial on 'fact boxes'. [europepmc]
- Exercise Interventions in Polycystic Ovary Syndrome: A Systematic Review and Meta-Analysis. [europepmc]
- The Potential of Functional Near-Infrared Spectroscopy-Based Neurofeedback-A Systematic Review and Recommendations for Best Practice. [europepmc]
- Anxiety and the Neurobiology of Temporally Uncertain Threat Anticipation. [europepmc]
- Semantic and cognitive tools to aid statistical science: replace confidence and significance by compatibility and surprise. [europepmc]
- Agreement between self-reported pre-pregnancy weight and measured first-trimester weight in Brazilian women. [europepmc]
- Opportunities for increased reproducibility and replicability of developmental neuroimaging. [europepmc]
- A systematic review of fMRI neurofeedback reporting and effects in clinical populations. [europepmc]
- The Practical Alternative to the p Value Is the Correctly Used p Value. [europepmc]
- Revealing taxon-specific heavy metal-resistance mechanisms in denitrifying phosphorus removal sludge using genome-centric metaproteomics. [europepmc]
- Estimating power in (generalized) linear mixed models: An open introduction and tutorial in R. [europepmc]
- Female (Under) Representation in Exercise Thermoregulation Research. [europepmc]
- Remote Work, Work Stress, and Work-Life during Pandemic Times: A Latin America Situation. [europepmc]
- Passive social media use and psychological well-being during the COVID-19 pandemic: The role of social comparison and emotion regulation. [europepmc]
- Individual variations in 'brain age' relate to early-life factors more than to longitudinal brain change. [europepmc]
- Global and regional prevalence of disabilities among children and adolescents: Analysis of findings from global health databases. [europepmc]
- Negativity drives online news consumption. [europepmc]
- Power to Detect What? Considerations for Planning and Evaluating Sample Size. [europepmc]
- The replication crisis has led to positive structural, procedural, and community changes. [europepmc]
- Screens, Teens, and Psychological Well-Being: Evidence From Three Time-Use-Diary Studies. [europepmc]
Discussions
Related