NDLI: Characterizing the limitations of using diagnosis codes in the context of machine learning for healthcare

Content Provider	Springer Nature : BioMed Central
Author	Guo, Lin Lawrence Morse, Keith E. Aftandilian, Catherine Steinberg, Ethan Fries, Jason Posada, Jose Fleming, Scott Lanyon Lemmon, Joshua Jessa, Karim Shah, Nigam Sung, Lillian
Abstract	Background Diagnostic codes are commonly used as inputs for clinical prediction models, to create labels for prediction tasks, and to identify cohorts for multicenter network studies. However, the coverage rates of diagnostic codes and their variability across institutions are underexplored. The primary objective was to describe lab- and diagnosis-based labels for 7 selected outcomes at three institutions. Secondary objectives were to describe agreement, sensitivity, and specificity of diagnosis-based labels against lab-based labels. Methods This study included three cohorts: SickKids from The Hospital for Sick Children, and StanfordPeds and StanfordAdults from Stanford Medicine. We included seven clinical outcomes with lab-based definitions: acute kidney injury, hyperkalemia, hypoglycemia, hyponatremia, anemia, neutropenia and thrombocytopenia. For each outcome, we created four lab-based labels (abnormal, mild, moderate and severe) based on test result and one diagnosis-based label. Proportion of admissions with a positive label were presented for each outcome stratified by cohort. Using lab-based labels as the gold standard, agreement using Cohen’s Kappa, sensitivity and specificity were calculated for each lab-based severity level. Results The number of admissions included were: SickKids (n = 59,298), StanfordPeds (n = 24,639) and StanfordAdults (n = 159,985). The proportion of admissions with a positive diagnosis-based label was significantly higher for StanfordPeds compared to SickKids across all outcomes, with odds ratio (99.9% confidence interval) for abnormal diagnosis-based label ranging from 2.2 (1.7–2.7) for neutropenia to 18.4 (10.1–33.4) for hyperkalemia. Lab-based labels were more similar by institution. When using lab-based labels as the gold standard, Cohen’s Kappa and sensitivity were lower at SickKids for all severity levels compared to StanfordPeds. Conclusions Across multiple outcomes, diagnosis codes were consistently different between the two pediatric institutions. This difference was not explained by differences in test results. These results may have implications for machine learning model development and deployment.
Related Links	https://bmcmedinformdecismak.biomedcentral.com/counter/pdf/10.1186/s12911-024-02449-8.pdf
Ending Page	9
Page Count	9
Starting Page	1
File Format	HTM / HTML
ISSN	14726947
DOI	10.1186/s12911-024-02449-8
Journal	BMC Medical Informatics and Decision Making
Issue Number	1
Volume Number	24
Language	English
Publisher	BioMed Central
Publisher Date	2024-02-14
Access Restriction	Open
Subject Keyword	Health Informatics Information Systems and Communication Service Management of Computing and Information Systems Electronic health records Diagnostic coding practice Cohort identification Outcome identification Machine learning for health
Content Type	Text
Resource Type	Article
Subject	Health Informatics Computer Science Applications Health Policy
Journal Impact Factor	3.3/2023
5-Year Journal Impact Factor	3.9/2023

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Characterizing the limitations of using diagnosis codes in the context of machine learning for healthcare

Neural-signature methods for structured EHR prediction

Prediction and diagnosis of depression using machine learning with electronic health records data: a systematic review

Anomaly-based threat detection in smart health using machine learning

Prediction of adverse pregnancy outcomes using machine learning techniques: evidence from analysis of electronic medical records data in Rwanda

Natural language processing data services for healthcare providers

DRG grouping by machine learning: from expert-oriented to data-based method

Evaluation of the accuracy of diagnostic coding for influenza compared to laboratory results: the availability of test results before hospital discharge facilitates improved coding accuracy

An explainable CNN approach for medical codes prediction from clinical text

Characterizing the limitations of using diagnosis codes in the context of machine learning for healthcare

Similar Documents

Characterizing the limitations of using diagnosis codes in the context of machine learning for healthcare

Neural-signature methods for structured EHR prediction

Prediction and diagnosis of depression using machine learning with electronic health records data: a systematic review

Anomaly-based threat detection in smart health using machine learning

Prediction of adverse pregnancy outcomes using machine learning techniques: evidence from analysis of electronic medical records data in Rwanda

Natural language processing data services for healthcare providers

DRG grouping by machine learning: from expert-oriented to data-based method

Evaluation of the accuracy of diagnostic coding for influenza compared to laboratory results: the availability of test results before hospital discharge facilitates improved coding accuracy

An explainable CNN approach for medical codes prediction from clinical text

Characterizing the limitations of using diagnosis codes in the context of machine learning for healthcare