2024/12/12 by Benno Uthayasooriyar, Uthayasooriyar, Benno, Antoine Ly +5 · 1 citation
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning in Healthcare #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2412.09341
openalex publication_date 2024/12/12 · openalex created_date 2024/12/14 · openalex updated_date 2026/07/28
Generic pre-trained neural networks may struggle to produce good results in specialized domains like finance and insurance. This is due to a domain mismatch between training data and downstream tasks, as in-domain data are often scarce due to privacy constraints. In this work, we compare different pre-training strategies for LayoutLM. We show that using domain-relevant documents improves results on a named-entity recognition (NER) problem using a novel dataset of anonymized insurance-related financial documents called Payslips. Moreover, we show that we can achieve competitive results using a smaller and faster model.