Exaggerated false positives by popular differential expression methods when analyzing human population samples

2022/03/15 by Yumei Li, Xinzhou Ge, Fanglue Peng +2 · 1 voice · 1 citation
Biochemistry, Genetics and Molecular Biology · #Gene expression and cancer classification #Molecular Biology Techniques and Applications #Cancer-related molecular mechanisms research

paper · pdf · doi:10.1186/s13059-022-02648-4

openalex publication_date 2022/03/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/03

Abstract

When identifying differentially expressed genes between two conditions using human population RNA-seq samples, we found a phenomenon by permutation analysis: two popular bioinformatics methods, DESeq2 and edgeR, have unexpectedly high false discovery rates. Expanding the analysis to limma-voom, NOISeq, dearseq, and Wilcoxon rank-sum test, we found that FDR control is often failed except for the Wilcoxon rank-sum test. Particularly, the actual FDRs of DESeq2 and edgeR sometimes exceed 20% when the target FDR is 5%. Based on these results, for population-level RNA-seq studies with large sample sizes, we recommend the Wilcoxon rank-sum test.

Citations

• On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other
• limma powers differential expression analyses for RNA-sequencing and microarray studies
• The Cancer Genome Atlas Pan-Cancer analysis project
• The Efficiency of Some Nonparametric Competitors of the t-Test
• GEPIA: a web server for cancer and normal gene expression profiling and interactive analyses
• The GTEx Consortium atlas of genetic regulatory effects across human tissues

Cited by

• Tool Choice Matters: Evaluating edgeR vs. DESeq2 for Sensitivity, Robustness, and Cross-Study Performance

Discussions

• Great paper: "When identifying differentially expressed genes ... we found ... [that] ... two popular bioinformatics methods, DESeq2 and edgeR, have unexpectedly high false discovery rates. ... we rec [bsky, 0 points, 0 comments]

Related

• Nullstrap-DE: A General Framework for Calibrating FDR and Preserving Power in DE Methods, with Applications to DESeq2 and edgeR
• Multiple Hypothesis Testing in Genomics
• Biases in differential expression analysis of RNA-seq data: A matter of replicate type
• Evaluation of tools for differential gene expression analysis by RNA-seq on a 48 biological replicate experiment
• Beware of counter-intuitive levels of false discoveries in datasets with strong intra-correlations