2025/02/20 by Hugo Sousa, Rúben Almeida, Sousa, Hugo +9 · 1 citation
Agricultural and Biological Sciences · Business, Management and Accounting · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Fermentation and Sensory Analysis #Sensory Analysis and Statistical Methods #Wine Industry and Tourism
paper · pdf · doi:10.48550/arxiv.2502.14394
openalex publication_date 2025/02/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Recent advances in natural language processing have raised expectations for generative models to produce coherent text across diverse language varieties. In the particular case of the Portuguese language, the predominance of Brazilian Portuguese corpora online introduces linguistic biases in these models, limiting their applicability outside of Brazil. To address this gap and promote the creation of European Portuguese resources, we developed a cross-domain language variety identifier (LVI) to discriminate between European and Brazilian Portuguese. Motivated by the findings of our literature review, we compiled the PtBrVarId corpus, a cross-domain LVI dataset, and study the effectiveness of transformer-based LVI classifiers for cross-domain scenarios. Although this research focuses on two Portuguese varieties, our contribution can be extended to other varieties and languages. We open source the code, corpus, and models to foster further research in this task.