2019/09/23 by Fábio Souza, Souza, Fábio, Rodrigo Nogueira +3 · 8 citations
Computer Science · Decision Sciences · #Data Quality and Management #Natural Language Processing Techniques #Topic Modeling #cs.CL #cs.IR #cs.LG
paper · pdf · doi:10.48550/arxiv.1909.10649
arxiv created 2020/02/27 · arxiv updated 2020/02/28
Recent advances in language representation using neural networks have made it viable to transfer the learned internal states of a trained model to downstream natural language processing tasks, such as named entity recognition (NER) and question answering. It has been shown that the leverage of pre-trained language models improves the overall performance on many tasks and is highly beneficial when labeled data is scarce. In this work, we train Portuguese BERT models and employ a BERT-CRF architecture to the NER task on the Portuguese language, combining the transfer capabilities of BERT with the structured predictions of CRF. We explore feature-based and fine-tuning training strategies for the BERT model. Our fine-tuning approach obtains new state-of-the-art results on the HAREM I dataset, improving the F1-score by 1 point on the selective scenario (5 NE classes) and by 4 points on the total scenario (10 NE classes).