NDLI: Drug knowledge discovery via multi-task learning and pre-trained models

Please wait, while we are loading the content...

Drug knowledge discovery via multi-task learning and pre-trained models

Content Provider	Springer Nature : BioMed Central
Author	Li, Dongfang Xiong, Ying Hu, Baotian Tang, Buzhou Peng, Weihua Chen, Qingcai
Abstract	Background Drug repurposing is to find new indications of approved drugs, which is essential for investigating new uses for approved or investigational drug efficiency. The active gene annotation corpus (named AGAC) is annotated by human experts, which was developed to support knowledge discovery for drug repurposing. The AGAC track of the BioNLP Open Shared Tasks using this corpus is organized by EMNLP-BioNLP 2019, where the “Selective annotation” attribution makes AGAC track more challenging than other traditional sequence labeling tasks. In this work, we show our methods for trigger word detection (Task 1) and its thematic role identification (Task 2) in the AGAC track. As a step forward to drug repurposing research, our work can also be applied to large-scale automatic extraction of medical text knowledge. Methods To meet the challenges of the two tasks, we consider Task 1 as the medical name entity recognition (NER), which cultivates molecular phenomena related to gene mutation. And we regard Task 2 as a relation extraction task, which captures the thematic roles between entities. In this work, we exploit pre-trained biomedical language representation models (e.g., BioBERT) in the information extraction pipeline for mutation-disease knowledge collection from PubMed. Moreover, we design the fine-tuning framework by using a multi-task learning technique and extra features. We further investigate different approaches to consolidate and transfer the knowledge from varying sources and illustrate the performance of our model on the AGAC corpus. Our approach is based on fine-tuned BERT, BioBERT, NCBI BERT, and ClinicalBERT using multi-task learning. Further experiments show the effectiveness of knowledge transformation and the ensemble integration of models of two tasks. We conduct a performance comparison of various algorithms. We also do an ablation study on the development set of Task 1 to examine the effectiveness of each component of our method. Results Compared with competitor methods, our model obtained the highest Precision (0.63), Recall (0.56), and F-score value (0.60) in Task 1, which ranks first place. It outperformed the baseline method provided by the organizers by 0.10 in F-score. The model shared the same encoding layers for the named entity recognition and relation extraction parts. And we obtained a second high F-score (0.25) in Task 2 with a simple but effective framework. Conclusions Experimental results on the benchmark annotation of genes with active mutation-centric function changes corpus show that integrating pre-trained biomedical language representation models (i.e., BERT, NCBI BERT, ClinicalBERT, BioBERT) into a pipe of information extraction methods with multi-task learning can improve the ability to collect mutation-disease knowledge from PubMed.
Related Links	https://bmcmedinformdecismak.biomedcentral.com/counter/pdf/10.1186/s12911-021-01614-7.pdf
Ending Page	9
Page Count	9
Starting Page	1
File Format	HTM / HTML
ISSN	14726947
DOI	10.1186/s12911-021-01614-7
Journal	BMC Medical Informatics and Decision Making
Issue Number	9
Volume Number	21
Language	English
Publisher	BioMed Central
Publisher Date	2021-11-16
Access Restriction	Open
Subject Keyword	Health Informatics Information Systems and Communication Service Management of Computing and Information Systems Gene mutation Drug repurposing Biomedical language models
Content Type	Text
Resource Type	Article
Subject	Health Informatics Computer Science Applications Health Policy
Journal Impact Factor	3.3/2023
5-Year Journal Impact Factor	3.9/2023

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Using word evolution to predict drug repurposing

A comparative study of pre-trained language models for named entity recognition in clinical trial eligibility criteria from multiple corpora

Path-based knowledge reasoning with textual semantic information for medical knowledge graph completion

Multi-task learning for Chinese clinical named entity recognition with external knowledge

Protein sequence analysis in the context of drug repurposing

Exploration of biomedical knowledge for recurrent glioblastoma using natural language processing deep learning models

Prediction of cognitive impairment via deep learning trained with multi-center neuropsychological test data

Semantic biomedical resource discovery: a Natural Language Processing framework

Deep learning with sentence embeddings pre-trained on biomedical corpora improves the performance of finding similar sentences in electronic medical records

Drug knowledge discovery via multi-task learning and pre-trained models

Similar Documents

Using word evolution to predict drug repurposing

A comparative study of pre-trained language models for named entity recognition in clinical trial eligibility criteria from multiple corpora

Path-based knowledge reasoning with textual semantic information for medical knowledge graph completion

Multi-task learning for Chinese clinical named entity recognition with external knowledge

Protein sequence analysis in the context of drug repurposing

Exploration of biomedical knowledge for recurrent glioblastoma using natural language processing deep learning models

Prediction of cognitive impairment via deep learning trained with multi-center neuropsychological test data

Semantic biomedical resource discovery: a Natural Language Processing framework

Deep learning with sentence embeddings pre-trained on biomedical corpora improves the performance of finding similar sentences in electronic medical records

Drug knowledge discovery via multi-task learning and pre-trained models