NDLI: Iterative guided machine learning-assisted systematic literature reviews: a diabetes case study

Content Provider	Springer Nature : BioMed Central
Author	Zimmerman, John Soler, Robin E. Lavinder, James Murphy, Sarah Atkins, Charisma Hulbert, LaShonda Lusk, Richard Ng, Boon Peng
Abstract	Background Systematic Reviews (SR), studies of studies, use a formal process to evaluate the quality of scientific literature and determine ensuing effectiveness from qualifying articles to establish consensus findings around a hypothesis. Their value is increasing as the conduct and publication of research and evaluation has expanded and the process of identifying key insights becomes more time consuming. Text analytics and machine learning (ML) techniques may help overcome this problem of scale while still maintaining the level of rigor expected of SRs. Methods In this article, we discuss an approach that uses existing examples of SRs to build and test a method for assisting the SR title and abstract pre-screening by reducing the initial pool of potential articles down to articles that meet inclusion criteria. Our approach differs from previous approaches to using ML as a SR tool in that it incorporates ML configurations guided by previously conducted SRs, and human confirmation on ML predictions of relevant articles during multiple iterative reviews on smaller tranches of citations. We applied the tailored method to a new SR review effort to validate performance. Results The case study test of the approach proved a sensitivity (recall) in finding relevant articles during down selection that may rival many traditional processes and show ability to overcome most type II errors. The study achieved a sensitivity of 99.5% (213 out of 214) of total relevant articles while only conducting a human review of 31% of total articles available for review. Conclusions We believe this iterative method can help overcome bias in initial ML model training by having humans reinforce ML models with new and relevant information, and is an applied step towards transfer learning for ML in SR.
Related Links	https://systematicreviewsjournal.biomedcentral.com/counter/pdf/10.1186/s13643-021-01640-6.pdf
Ending Page	8
Page Count	8
Starting Page	1
File Format	HTM / HTML
ISSN	20464053
DOI	10.1186/s13643-021-01640-6
Journal	Systematic Reviews
Issue Number	1
Volume Number	10
Language	English
Publisher	BioMed Central
Publisher Date	2021-04-02
Access Restriction	Open
Subject Keyword	Medicine Public Health Biomedicine Statistics for Life Sciences Health Sciences Machine learning Systematic review screening Natural language processing Transfer learning Machine learning configurations Applied case study Medicine/Public Health
Content Type	Text
Resource Type	Article
Subject	Medicine
Journal Impact Factor	6.3/2023
5-Year Journal Impact Factor	4.5/2023

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Semi-automated title-abstract screening using natural language processing and machine learning

Assessing the accuracy of machine-assisted abstract screening with DistillerAI: a user study

Title and abstract screening for literature reviews using large language models: an exploratory study in the biomedical domain

Research Screener: a machine learning tool to semi-automate abstract screening for systematic reviews

Automation of systematic reviews of biomedical literature: a scoping review of studies indexed in PubMed

Ensemble of deep learning language models to support the creation of living systematic reviews for the COVID-19 literature

Technology-assisted title and abstract screening for systematic reviews: a retrospective evaluation of the Abstrackr machine learning tool

Aligning text mining and machine learning algorithms with best practices for study selection in systematic literature reviews

Performance and usability of machine learning for screening in systematic reviews: a comparative evaluation of three tools

Iterative guided machine learning-assisted systematic literature reviews: a diabetes case study

Similar Documents

Semi-automated title-abstract screening using natural language processing and machine learning

Assessing the accuracy of machine-assisted abstract screening with DistillerAI: a user study

Title and abstract screening for literature reviews using large language models: an exploratory study in the biomedical domain

Research Screener: a machine learning tool to semi-automate abstract screening for systematic reviews

Automation of systematic reviews of biomedical literature: a scoping review of studies indexed in PubMed

Ensemble of deep learning language models to support the creation of living systematic reviews for the COVID-19 literature

Technology-assisted title and abstract screening for systematic reviews: a retrospective evaluation of the Abstrackr machine learning tool

Aligning text mining and machine learning algorithms with best practices for study selection in systematic literature reviews

Performance and usability of machine learning for screening in systematic reviews: a comparative evaluation of three tools

Iterative guided machine learning-assisted systematic literature reviews: a diabetes case study