NDLI: Parsing clinical text: how good are the state-of-the-art parsers?

Content Provider	Springer Nature : BioMed Central
Author	Jiang, Min Huang, Yang Fan, Jung-wei Tang, Buzhou Denny, Josh Xu, Hua
Abstract	Background Parsing, which generates a syntactic structure of a sentence (a parse tree), is a critical component of natural language processing (NLP) research in any domain including medicine. Although parsers developed in the general English domain, such as the Stanford parser, have been applied to clinical text, there are no formal evaluations and comparisons of their performance in the medical domain. Methods In this study, we investigated the performance of three state-of-the-art parsers: the Stanford parser, the Bikel parser, and the Charniak parser, using following two datasets: (1) A Treebank containing 1,100 sentences that were randomly selected from progress notes used in the 2010 i2b2 NLP challenge and manually annotated according to a Penn Treebank based guideline; and (2) the MiPACQ Treebank, which is developed based on pathology notes and clinical notes, containing 13,091 sentences. We conducted three experiments on both datasets. First, we measured the performance of the three state-of-the-art parsers on the clinical Treebanks with their default settings. Then we re-trained the parsers using the clinical Treebanks and evaluated their performance using the 10-fold cross validation method. Finally we re-trained the parsers by combining the clinical Treebanks with the Penn Treebank. Results Our results showed that the original parsers achieved lower performance in clinical text (Bracketing F-measure in the range of 66.6%-70.3%) compared to general English text. After retraining on the clinical Treebank, all parsers achieved better performance, with the best performance from the Stanford parser that reached the highest Bracketing F-measure of 73.68% on progress notes and 83.72% on the MiPACQ corpus using 10-fold cross validation. When the combined clinical Treebanks and Penn Treebank was used, of the three parsers, the Charniak parser achieved the highest Bracketing F-measure of 73.53% on progress notes and the Stanford parser reached the highest F-measure of 84.15% on the MiPACQ corpus. Conclusions Our study demonstrates that re-training using clinical Treebanks is critical for improving general English parsers' performance on clinical text, and combining clinical and open domain corpora might achieve optimal performance for parsing clinical text.
Related Links	https://bmcmedinformdecismak.biomedcentral.com/counter/pdf/10.1186/1472-6947-15-S1-S2.pdf
Ending Page	6
Page Count	6
Starting Page	1
File Format	HTM / HTML
ISSN	14726947
DOI	10.1186/1472-6947-15-S1-S2
Journal	BMC Medical Informatics and Decision Making
Issue Number	1
Volume Number	15
Language	English
Publisher	BioMed Central
Publisher Date	2015-05-20
Access Restriction	Open
Subject Keyword	Health Informatics Information Systems and Communication Service Management of Computing and Information Systems Medical language processing natural language processing parsing clinical text NLP
Content Type	Text
Resource Type	Article
Subject	Health Informatics Computer Science Applications Health Policy
Journal Impact Factor	3.3/2023
5-Year Journal Impact Factor	3.9/2023

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Parsing Clinical Text: How Good are the State-of-the-Art Parsers?

Parsing clinical text: how good are the state-of-the-art parsers?

Parsing clinical text using the state-of-the-art deep learning based parsers: a systematic comparison

NLP modeling recommendations for restricted data availability in clinical settings

Extracting health-related causality from twitter messages using natural language processing

Editorial: The second international workshop on health natural language processing (HealthNLP 2019)

Case-based medical informatics

Automation of a problem list using natural language processing

Ontology-based venous thromboembolism risk assessment model developing from medical records

Parsing clinical text: how good are the state-of-the-art parsers?

Similar Documents

Parsing Clinical Text: How Good are the State-of-the-Art Parsers?

Parsing clinical text: how good are the state-of-the-art parsers?

Parsing clinical text using the state-of-the-art deep learning based parsers: a systematic comparison

NLP modeling recommendations for restricted data availability in clinical settings

Extracting health-related causality from twitter messages using natural language processing

Editorial: The second international workshop on health natural language processing (HealthNLP 2019)

Case-based medical informatics

Automation of a problem list using natural language processing

Ontology-based venous thromboembolism risk assessment model developing from medical records

Parsing clinical text: how good are the state-of-the-art parsers?