Theory-Testing in Psychology and Physics: A Methodological Paradox

1967/06/01 by Paul E. Meehl · 31 citations
Decision Sciences · Psychology · #Meta-analysis and systematic reviews #Psychometric Methodologies and Testing #Evolutionary Psychology and Human Behavior

paper · doi:10.1086/288135

Abstract

Because physical theories typically predict numerical values, an improvement in experimental precision reduces the tolerance range and hence increases corroborability. In most psychological research, improved power of a statistical design leads to a prior probability approaching ½ of finding a significant difference in the theoretically predicted direction. Hence the corroboration yielded by “success” is very weak, and becomes weaker with increased precision. “Statistical significance” plays a logical role in psychology precisely the reverse of its role in physics. This problem is worsened by certain unhealthy tendencies prevalent among psychologists, such as a premium placed on experimental “cuteness” and a free reliance upon ad hoc explanations to avoid refutation.

Cited by

• The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective
• The Problem of Units and the Circumstance for POMP
• Toward a More Powerful Experimental Communication Science: An Assessment of Two Decades’ Research (2001–2023)
• Experimental education research: rethinking why, how, and when to use random assignment
• The Recurrence of Fundamental Questions: A Historical and Philosophical Analysis of Major Disciplinary Crises in Psychology
• Elegant theories and the problems of social psychology
• Privacy calculus, privacy paradox, and context collapse: A replication of three key studies in communication privacy research
• Nonsignificance misinterpreted as an effect’s absence in psychology: prevalence and temporal analyses
• There are no shortcuts to theory
• Why Hypothesis Testers Should Spend Less Time Testing Hypotheses
• Generalizability of choice architecture interventions
• Three-Sided Testing to Establish Practical Significance: A Tutorial
• From Hypotheses to Systematicity and Back
• Bayesian estimation supersedes the t test.
• There are no shortcuts to theory
• Alternative to the statistical mass confusion of testing for “no effect”
• Beyond binary group categorization: towards a dynamic view of human groups
• Integrating Media Selection and Media Effects Using Decision Theory
• The Theory Crisis in Psychology: How to Move Forward
• Falsification and the Methodology of Scientific Research Programmes
• The generalizability crisis
• A possible disconnect in JDM research articles: <i>Results</i> sections display an estimation perspective while <i>Discussion</i> sections belie an estimation mindset
• The use and interpretation of unstandardized and standardized effect sizes in psychology: current practices and challenges
• Understanding and predicting cultural change
• The institutional impact of research challenges and constraints on psychology and other social and behavioral sciences
• 281. MEVATRON77DX67 における電子線用可変型絞りについて(治療-6 電子線)
• Bayesian data analysis for newcomers
• When expert predictions fail
• Replicability Crisis in Social Psychology: Looking at the Past to Find New Pathways for the Future
• Humanist Interpretation Versus Coding Text Samples
• Replicability: Terminology, Measuring Success, and Strategy

Related

• On the Logic (plus some history and philosophy) of Statistical Tests and Scientific Investigation
• The state and status of theory in psychological science
• Why Hypothesis Testers Should Spend Less Time Testing Hypotheses
• Theory before the test: How to build high-verisimilitude explanatory theories in psychological science
• Fostering transparency and reproducibility in psychological science.