2024/08/30 by Sujoy Roychowdhury, Roychowdhury, Sujoy, Sumit Soman +8 · 1 citation
Computer Science · #68T50 #Advanced Text Analysis Techniques #FOS: Computer and information sciences #I.2.7 #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Service-Oriented Architecture and Web Services #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2408.17008
openalex publication_date 2024/08/30 · openalex created_date 2024/10/19 · openalex updated_date 2026/07/28
With the ubiquitous use of document corpora for question answering, one important aspect which is especially relevant for technical documents is the ability to extract information from tables which are interspersed with text. The major challenge in this is that unlike free-flow text or isolated set of tables, the representation of a table in terms of what is a relevant chunk is not obvious. We conduct a series of experiments examining various representations of tabular data interspersed with text to understand the relative benefits of different representations. We choose a corpus of 3rd Generation Partnership Project (3GPP) documents since they are heavily interspersed with tables. We create expert curated dataset of question answers to evaluate our approach. We conclude that row level representations with corresponding table header information being included in every cell improves the performance of the retrieval, thus leveraging the structural information present in the tabular data.