XOR QA: Cross-lingual Open-Retrieval Question Answering

2020/10/22 by Akari Asai, Jungo Kasai, Asai, Akari +9 · 22 citations
Computer Science · #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.2010.11856

Published as a conference paper at NAACL-HLT 2021 (long)

arxiv created 2021/04/13 · arxiv updated 2021/04/14

Abstract

Multilingual question answering tasks typically assume answers exist in the same language as the question. Yet in practice, many languages face both information scarcity -- where languages have few reference articles -- and information asymmetry -- where questions reference concepts from other cultures. This work extends open-retrieval question answering to a cross-lingual setting enabling questions from one language to be answered via answer content from another language. We construct a large-scale dataset built on questions from TyDi QA lacking same-language answers. Our task formulation, called Cross-lingual Open Retrieval Question Answering (XOR QA), includes 40k information-seeking questions from across 7 diverse non-English languages. Based on this dataset, we introduce three new tasks that involve cross-lingual document retrieval using multi-lingual and English resources. We establish baselines with state-of-the-art machine translation systems and cross-lingual pretrained models. Experimental results suggest that XOR QA is a challenging task that will facilitate the development of novel techniques for multilingual question answering. Our data and code are available at https://nlp.cs.washington.edu/xorqa.

Citations

• Datasheets for Datasets

Cited by

• Language-Coupled Reinforcement Learning for Multilingual Retrieval-Augmented Generation
• M4-RAG: A Massive-Scale Multilingual Multi-Cultural Multimodal RAG
• Generative Query Expansion with Multilingual LLMs for Cross-Lingual Information Retrieval
• RAGSmith: A Framework for Finding the Optimal Composition of Retrieval-Augmented Generation Methods Across Datasets
• LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMs
• Quality-Aware Translation Tagging in Multilingual RAG system
• EmbeddingGemma: Powerful and Lightweight Text Representations
• The Medium Is Not the Message: Deconfounding Text Embeddings via Linear Concept Erasure
• A Survey of Long-Document Retrieval in the PLM and LLM Era
• MegaWika 2: A More Comprehensive Multilingual Collection of Articles and their Sources
• The Cross-Lingual Cost: Retrieval Biases in RAG over Arabic-English Corpora
• Eka-Eval: An Evaluation Framework for Low-Resource Multilingual Large Language Models
• The Medium Is Not the Message: Deconfounding Document Embeddings via Linear Concept Erasure
• WikiGap: Promoting Epistemic Equity by Surfacing Knowledge Gaps Between English Wikipedia and other Language Editions
• OWL: Probing Cross-Lingual Recall of Memorized Texts via World Literature
• MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs
• MiLQ: Benchmarking IR Models for Bilingual Web Search with Mixed Language Queries
• XRAG: Cross-lingual Retrieval-Augmented Generation
• Improving Multilingual Retrieval-Augmented Language Models through Dialectic Reasoning Argumentations
• CLIRudit: Cross-Lingual Information Retrieval of Scientific Documents
• Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Task
• FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents

Related

• One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval
• MIA 2022 Shared Task: Evaluating Cross-lingual Open-Retrieval Question Answering for 16 Diverse Languages
• Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Task
• Investigating Information Inconsistency in Multilingual Open-Domain Question Answering
• ZusammenQA: Data Augmentation with Specialized Models for Cross-lingual Open-retrieval Question Answering System