New Methods & Metrics for LFQA tasks

2021/12/26 by Mahapatra, Suchismit, Blagojevic, Vladimir, Bertorello, Pablo +1 · 1 citation
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2112.13432

Abstract

Long-form question answering (LFQA) tasks require retrieving the documents pertinent to a query, using them to form a paragraph-length answer. Despite considerable progress in LFQA modeling, fundamental issues impede its progress: i) train/validation/test dataset overlap, ii) absence of automatic metrics and iii) generated answers not being "grounded" in retrieved documents. This work addresses every one these critical bottlenecks, contributing natural language inference/generation (NLI/NLG) methods and metrics that make significant strides to their alleviation.

Cited by

• AI-Powered Annotation Pipelines for Stabilizing Large Language Models: A Human-AI Synergy Approach

Related

• Hurdles to Progress in Long-form Question Answering
• Long-form Question Answering: An Iterative Planning-Retrieval-Generation Approach
• Read before Generate! Faithful Long Form Question Answering with Machine Reading
• Generative Long-form Question Answering: Relevance, Faithfulness and Succinctness
• Query Refinement Prompts for Closed-Book Long-Form Question Answering