A survey of diversity quantification in natural language processing: The why, what, where and how
2025/07/28 by Estève, Louis, de Marneffe, Marie-Catherine, Melnik, Nurit +2
#Computation and Language (cs.CL) #FOS: Computer and information sciences
paper · doi:10.48550/arxiv.2507.20858
Abstract
The concept of diversity has received increased consideration in Natural Language Processing (NLP) in recent years. This is due to various motivations like promoting and inclusion, approximating human linguistic behavior, and increasing systems' performance. Diversity has however often been addressed in an ad hoc manner in NLP, and with few explicit links to other domains where this notion is better theorized. We survey articles in the ACL Anthology from the past 6 years, with "diversity" or "diverse" in their title. We find a wide range of settings in which diversity is quantified, often highly specialized and using inconsistent terminology. We put forward a unified taxonomy of why, what on, where, and how diversity is measured in NLP. Diversity measures are cast upon a unified framework from ecology and economy (Stirling, 2007) with 3 dimensions of diversity: variety, balance and disparity. We discuss the trends which emerge due to this systematized approach. We believe that this study paves the way towards a better formalization of diversity in NLP, which should bring a better understanding of this notion and a better comparability between various approaches.
Citations
- Evaluating the Evaluation of Diversity in Commonsense Generation
- Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric
- Benchmarking Linguistic Diversity of Large Language Models
- Hallucination Diversity-Aware Active Learning for Text Summarization
- Improving Content Recommendation: Knowledge Graph-Based Semantic Contrastive Learning for Diversity and Cold-Start Users
- Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment
- A Measure for Transparent Comparison of Linguistic Diversity in Multilingual NLP Data Sets
- Fine-Grained Natural Language Inference Based Faithfulness Evaluation for Diverse Summarisation Tasks
- Generating Diverse Translation with Perturbed kNN-MT
- Partially Randomizing Transformer Weights for Dialogue Response Diversity
- The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text
- JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processing
- Fair Abstractive Summarization of Diverse Perspectives
- Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Voting
- GD-COMET: A Geo-Diverse Commonsense Inference Model
- Fidelity-Enriched Contrastive Search: Reconciling the Faithfulness-Diversity Trade-Off in Text Generation
- Prefix-diffusion: A Lightweight Diffusion Model for Diverse Image Captioning
- Measuring Lexical Diversity in Texts: The Twofold Length Problem
- Towards Diverse and Effective Question-Answer Pair Generation from Children Storybooks
- DLAMA: A Framework for Curating Culturally Diverse Facts for Probing the Knowledge of Pretrained Language Models
- Examining Bias in Opinion Summarisation Through the Perspective of Opinion Diversity
- Diversity-Aware Coherence Loss for Improving Neural Topic Models
- Understanding Compositional Data Augmentation in Typologically Diverse Morphological Inflection
- ChatGPT to Replace Crowdsourcing of Paraphrases for Intent Classification: Higher Diversity and Comparable Model Robustness
- LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
- Why Can't Discourse Parsing Generalize? A Thorough Investigation of the Impact of Data Diversity
- PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism
- ArtELingo: A Million Emotion Annotations of WikiArt with Emphasis on Diversity over Language and Culture
- UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization
- Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attention
- MINION: a Large-Scale and Diverse Dataset for Multilingual Event Detection
- Diverse Parallel Data Synthesis for Cross-Database Adaptation of Text-to-SQL Parsers
- Recurrence Boosts Diversity! Revisiting Recurrent Latent Variable in Transformer-Based Variational AutoEncoder for Diverse Text Generation
- Measuring and Improving Semantic Diversity of Dialogue Generation
- MIA 2022 Shared Task: Evaluating Cross-lingual Open-Retrieval Question Answering for 16 Diverse Languages
- "Diversity and Uncertainty in Moderation" are the Key to Data Selection for Multilingual Few-shot Transfer
- One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation
- Evaluating the Diversity, Equity and Inclusion of NLP Technology: A Case Study for Indian Languages
- Diverse Lottery Tickets Boost Ensemble from a Single Pretrained Model
- ParaCotta: Synthetic Multilingual Paraphrase Corpora from the Most Diverse Translation Sample Pair
- Semantic Diversity in Dialogue with Natural Language Inference
- A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generation
- IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languages
- Language Diversity: Visible to Humans, Exploitable by Machines
- Improving the Diversity of Unsupervised Paraphrasing with Embedding Outputs
- Different Strokes for Different Folks: Investigating Appropriate Further\n Pre-training Approaches for Diverse Dialogue Tasks
- Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning
- Towards Evaluation of Cross-document Coreference Resolution Models Using Datasets with Diverse Annotation Schemes
- Lingxi: A Diversity-aware Chinese Modern Poetry Generation System
- Generating Diverse Descriptions from Semantic Graphs
- Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training
- ProtAugment: Unsupervised diverse short-texts paraphrasing for intent detection meta-learning
- One2Set: Generating Diverse Keyphrases as a Set
- Automatic Classification of Human Translation and Machine Translation: A Study from the Perspective of Lexical Diversity
- TextBox: A Unified, Modularized, and Extensible Framework for Text Generation
- Event Detection: Gate Diversity and Syntactic Importance Scoresfor Graph Convolution Neural Networks
- Generating Diverse Translation from Model Distribution with Dropout
- COD3S: Diverse Generation with Discrete Semantic Signatures
- Paraphrase Generation as Zero-Shot Multilingual Translation: Disentangling Semantic Similarity from Lexical and Syntactic Diversity
- MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering
- A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers
- SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection
- Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs
- Trading Off Diversity and Quality in Natural Language Generation
- Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation
- Evaluating the Evaluation of Diversity in Natural Language Generation
- The State and Fate of Linguistic Diversity and Inclusion in the NLP\n World
- Diversity, Density, and Homogeneity: Quantitative Characteristic Metrics for Text Collections
- Measuring Diversity in Heterogeneous Information Networks
- Multi-Head Attention with Diversity for Learning Grounded Multilingual Multimodal Representations
- Generating Diverse Story Continuations with Controllable Semantics
- Improving Generative Visual Dialog by Answering Diverse Questions
- Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset
- Comparison of Diverse Decoding Methods from Conditional Language Models
- BERTScore: Evaluating Text Generation with BERT
- Jointly Measuring Diversity and Quality in Text Generation Models
- Outlier Detection for Improved Data Quality and Diversity in Dialog Systems
- Jointly Optimizing Diversity and Relevance in Neural Response Generation
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
- Interdisciplinarity as Diversity in Citation Patterns among Journals:\n Rao-Stirling Diversity, Relative Variety, and the Gini coefficient
- Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing
- Texygen: A Benchmarking Platform for Text Generation Models
- DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
- A Diversity-Promoting Objective Function for Neural Conversation Models
- Diversity in Spectral Learning for Natural Language Parsing
- Measuring diversity: the importance of species similarity
- Functional diversity measures: an overview of their redundancy and their ability to discriminate community assembly rules
- Where are your Manners? Sharing Best Community Practices in the Web 2.0
- <i>Entropy and diversity</i>
Related