Theory-Testing in Psychology and Physics: A Methodological Paradox
1967/06/01 by Paul E. Meehl · 996 citations
Decision Sciences · Mathematics · Psychology · #Cognitive psychology #Computer science #Econometrics #Epistemology #Evolutionary Psychology and Human Behavior #Mathematical economics #Mathematics #Meta-analysis and systematic reviews #Outcome (game theory) #Philosophy #Psychological research #Psychology #Psychometric Methodologies and Testing #Range (aeronautics) #Social psychology #Statistical hypothesis testing #Statistical power #Statistics
paper · doi:10.1086/288135
published in Philosophy of Science 34(2), 103-115 (Cambridge University Press)
openalex publication_date 1967/06/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Abstract
Because physical theories typically predict numerical values, an improvement in experimental precision reduces the tolerance range and hence increases corroborability. In most psychological research, improved power of a statistical design leads to a prior probability approaching ½ of finding a significant difference in the theoretically predicted direction. Hence the corroboration yielded by “success” is very weak, and becomes weaker with increased precision. “Statistical significance” plays a logical role in psychology precisely the reverse of its role in physics. This problem is worsened by certain unhealthy tendencies prevalent among psychologists, such as a premium placed on experimental “cuteness” and a free reliance upon ad hoc explanations to avoid refutation.
Cited by
- The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective
- The Problem of Units and the Circumstance for POMP
- Toward a More Powerful Experimental Communication Science: An Assessment of Two Decades’ Research (2001–2023)
- Experimental education research: rethinking why, how, and when to use random assignment
- The Recurrence of Fundamental Questions: A Historical and Philosophical Analysis of Major Disciplinary Crises in Psychology
- Elegant theories and the problems of social psychology
- Privacy calculus, privacy paradox, and context collapse: A replication of three key studies in communication privacy research
- Nonsignificance misinterpreted as an effect’s absence in psychology: prevalence and temporal analyses
- There are no shortcuts to theory
- Why Hypothesis Testers Should Spend Less Time Testing Hypotheses
- Generalizability of choice architecture interventions
- Three-Sided Testing to Establish Practical Significance: A Tutorial
- From Hypotheses to Systematicity and Back
- Bayesian estimation supersedes the t test.
- There are no shortcuts to theory
- Alternative to the statistical mass confusion of testing for “no effect”
- Beyond binary group categorization: towards a dynamic view of human groups
- Integrating Media Selection and Media Effects Using Decision Theory
- The Theory Crisis in Psychology: How to Move Forward
- Falsification and the Methodology of Scientific Research Programmes
- The generalizability crisis
- A possible disconnect in JDM research articles: Results sections display an estimation perspective while Discussion sections belie an estimation mindset
- The use and interpretation of unstandardized and standardized effect sizes in psychology: current practices and challenges
- Understanding and predicting cultural change
- The institutional impact of research challenges and constraints on psychology and other social and behavioral sciences
- 281. MEVATRON77DX67 における電子線用可変型絞りについて(治療-6 電子線)
- Bayesian data analysis for newcomers
- When expert predictions fail
- Replicability Crisis in Social Psychology: Looking at the Past to Find New Pathways for the Future
- Humanist Interpretation Versus Coding Text Samples
- Replicability: Terminology, Measuring Success, and Strategy
- Appraising and Amending Theories: The Strategy of Lakatosian Defense and Two Principles that Warrant It
- Simulation-Based Power Analyses for the Smallest Effect Size of Interest: A Confidence-Interval Approach for Minimum-Effect and Equivalence Testing
- Productive Explanation: A Framework for Evaluating Explanations in Psychological Science
- Meta-Analytic Associations Between the Student-Teacher Relationship Scale and Students’ Social Competence With Peers
- The Integration of Continuous and Discrete Latent Variable Models: Potential Problems and Promising Opportunities.
- Measuring Psychological Change during Cognitive Behaviour Therapy in Primary Care: A Polish Study Using ‘PSYCHLOPS’ (Psychological Outcome Profiles)
- Promises and Perils of Experimentation: The Mutual-Internal-Validity Problem
- How Do I Know What My Theory Predicts?
- A Meta-Analysis of Personality in Scientific and Artistic Creativity
- Cognitive-behavioral therapy versus other therapies: Redux
- Towards a balanced social psychology: Causes, consequences, and cures for the problem-seeking approach to social behavior and cognition
- The earth is round (p < .05).
- Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology.
- Factors and Taxa, Traits and Types, Differences of Degree and Differences in Kind
- Specific Genetic Etiology, Psychodynamics, and Therapeutic Nihilism
- Toward an Integrated Theory of Schizotaxia, Schizotypy, and Schizophrenia
- Why Summaries of Research on Psychological Theories are Often Uninterpretable
- A critical review of the neuroimaging literature on synesthesia
- Paul E. Meehl [wikipedia]
- Statistical inferences under the Null hypothesis: common mistakes and pitfalls in neuroimaging studies. [europepmc]
- A critical review of the neuroimaging literature on synesthesia. [europepmc]
- The significance fallacy in inferential statistics. [europepmc]
- The natural selection of bad science. [europepmc]
- A Tutorial on Hunting Statistical Significance by Chasing N . [europepmc]
- The earth is flat ( p > 0.05): significance thresholds and the crisis of unreplicable research. [europepmc]
- When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment. [europepmc]
- From Discovery to Justification: Outline of an Ideal Research Program in Empirical Psychology. [europepmc]
- Manipulating the Alpha Level Cannot Cure Significance Testing. [europepmc]
- From a Sampling Precision Perspective, Skewness Is a Friend and Not an Enemy! [europepmc]
- Establishment of Best Practices for Evidence for Prediction: A Review. [europepmc]
- ENIGMA brain injury: Framework, challenges, and opportunities. [europepmc]
- Open Science in Education Sciences. [europepmc]
- Does Postmodernism Really Entail a Disregard for the Truth? Similarities and Differences in Postmodern and Critical Rationalist Conceptualizations of Truth, Progress, and Empirical Research Methods. [europepmc]
- Machine Learning Assessment of Early Life Factors Predicting Suicide Attempt in Adolescence or Young Adulthood. [europepmc]
- New relevance and significance measures to replace p-values. [europepmc]
- Preregistration in diverse contexts: a preregistration template for the application of cognitive models. [europepmc]
- Providing Evidence for the Null Hypothesis in Functional Magnetic Resonance Imaging Using Group-Level Bayesian Inference. [europepmc]
- Restoration of Attention by Rest in a Multitasking World: Theory, Methodology, and Empirical Evidence. [europepmc]
- Assessing Measurement Invariance Across Multiple Groups: When Is Fit Good Enough? [europepmc]
- Averting the Next Credibility Crisis in Psychological Science: Within-Person Methods for Personalized Diagnostics and Intervention. [europepmc]
- I'm not sure that curve means what you think it means: Toward a [more] realistic understanding of the role of eye-movement generation in the Visual World Paradigm. [europepmc]
- Linking computational models of two core tasks of cognitive control. [europepmc]
- Better Inference in Neuroscience: Test Less, Estimate More. [europepmc]
- Published correlational effect sizes in social and developmental psychology. [europepmc]
Related