Text Mining Drug/Chemical-Protein Interactions using an Ensemble of BERT and T5 Based Models

2021/11/30 by Virginia Adams, Adams, Virginia, Hoo-Chang Shin +8
Biochemistry, Genetics and Molecular Biology · Computer Science · #Biomedical Text Mining and Ontologies #Chemical Synthesis and Analysis #Computation and Language (cs.CL) #Computational Drug Discovery Methods #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2111.15617

openalex publication_date 2021/11/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In Track-1 of the BioCreative VII Challenge participants are asked to identify interactions between drugs/chemicals and proteins. In-context named entity annotations for each drug/chemical and protein are provided and one of fourteen different interactions must be automatically predicted. For this relation extraction task, we attempt both a BERT-based sentence classification approach, and a more novel text-to-text approach using a T5 model. We find that larger BERT-based models perform better in general, with our BioMegatron-based model achieving the highest scores across all metrics, achieving 0.74 F1 score. Though our novel T5 text-to-text method did not perform as well as most of our BERT-based models, it outperformed those trained on similar data, showing promising results, achieving 0.65 F1 score. We believe a text-to-text approach to relation extraction has some competitive advantages and there is a lot of room for research advancement.

Related

• CU-UD: text-mining drug and chemical-protein interactions with ensembles of BERT-based models
• Chemical-protein relation extraction with ensembles of SVM, CNN, and RNN models
• BERTChem-DDI : Improved Drug-Drug Interaction Prediction from text using Chemical Structure Information
• Chemical-protein Interaction Extraction via Gaussian Probability Distribution and External Biomedical Knowledge
• Evaluation of GPT and BERT-based models on identifying protein-protein interactions in biomedical text