2017/01/29 by Ebrahim Ansari, M. H. Sadreddini, Ansari, Ebrahim +8
Computer Science · #Authorship Attribution and Profiling #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling #cs.CL
paper · pdf · doi:10.48550/arxiv.1701.08339
30 pages, Accepted to be published in "Applications of Comparable Corpora", Berlin: Language Science Press
arxiv created 2017/01/29 · openalex publication_date 2017/01/29 · arxiv updated 2017/01/31 · openalex created_date 2017/02/10 · openalex updated_date 2026/07/28
The effectiveness of a statistical machine translation system (SMT) is very dependent upon the amount of parallel corpus used in the training phase. For low-resource language pairs there are not enough parallel corpora to build an accurate SMT. In this paper, a novel approach is presented to extract bilingual Persian-Italian parallel sentences from a non-parallel (comparable) corpus. In this study, English is used as the pivot language to compute the matching scores between source and target sentences and candidate selection phase. Additionally, a new monolingual sentence similarity metric, Normalized Google Distance (NGD) is proposed to improve the matching process. Moreover, some extensions of the baseline system are applied to improve the quality of extracted sentences measured with BLEU. Experimental results show that using the new pivot based extraction can increase the quality of bilingual corpus significantly and consequently improves the performance of the Persian-Italian SMT system.