Appendix - Recommended Statistical Significance Tests for NLP Tasks

2018/09/05 by Rotem Dror, Roi Reichart, Dror, Rotem +1
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1809.01448

openalex publication_date 2018/09/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Statistical significance testing plays an important role when drawing conclusions from experimental results in NLP papers. Particularly, it is a valuable tool when one would like to establish the superiority of one algorithm over another. This appendix complements the guide for testing statistical significance in NLP presented in \citedror2018hitchhiker by proposing valid statistical tests for the common tasks and evaluation measures in the field.

Related

• More accurate tests for the statistical significance of result differences
• deep-significance - Easy and Meaningful Statistical Significance Testing in the Age of Neural Networks
• Please, Don't Forget the Difference and the Confidence Interval when Seeking for the State-of-the-Art Status
• When More Is Less: Pitfalls of significance testing
• Drawing Causal Inferences About Performance Effects in NLP