When to Use What: An In-Depth Comparative Empirical Analysis of OpenIE Systems for Downstream Applications

2022/11/15 by Kevin Pei, Pei, Kevin, Ishan Jindal +7 · 1 citation
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Software Engineering Research #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2211.08228

openalex publication_date 2022/11/15 · openalex created_date 2022/11/24 · openalex updated_date 2026/07/28

Abstract

Open Information Extraction (OpenIE) has been used in the pipelines of various NLP tasks. Unfortunately, there is no clear consensus on which models to use in which tasks. Muddying things further is the lack of comparisons that take differing training sets into account. In this paper, we present an application-focused empirical survey of neural OpenIE models, training sets, and benchmarks in an effort to help users choose the most suitable OpenIE systems for their applications. We find that the different assumptions made by different models and datasets have a statistically significant effect on performance, making it important to choose the most appropriate model for one's applications. We demonstrate the applicability of our recommendations on a downstream Complex QA application.

Cited by

• π-CoT: Prolog-Initialized Chain-of-Thought Prompting for Multi-Hop Question-Answering

Related

• A Survey on Open Information Extraction from Rule-based Model to Large Language Model
• A Survey on Neural Open Information Extraction: Current Status and Future Directions
• Analysing Errors of Open Information Extraction Systems
• OpenIE6: Iterative Grid Labeling and Coordination Analysis for Open\n Information Extraction
• IELM: An Open Information Extraction Benchmark for Pre-Trained Language Models