NDLI: Interpretability, credibility, and usability of hospital-specific template matching versus regression-based hospital performance assessments; a multiple methods study

Content Provider	Springer Nature : BioMed Central
Author	McGrath, Brenda M. Takamine, Linda Hogan, Cainnear K. Hofer, Timothy P. Rosen, Amy K. Sussman, Jeremy B. Wiitala, Wyndy L. Ryan, Andrew M. Prescott, Hallie C.
Abstract	Background Hospital-specific template matching (HS-TM) is a newer method of hospital performance assessment. Objective To assess the interpretability, credibility, and usability of HS-TM-based vs. regression-based performance assessments. Research design We surveyed hospital leaders (January-May 2021) and completed follow-up semi-structured interviews. Surveys included four hypothetical performance assessment vignettes, with method (HS-TM, regression) and hospital mortality randomized. Subjects Nationwide Veterans Affairs Chiefs of Staff, Medicine, and Hospital Medicine. Measures Correct interpretation; self-rated confidence in interpretation; and self-rated trust in assessment (via survey). Concerns about credibility and main uses (via thematic analysis of interview transcripts). Results In total, 84 participants completed 295 survey vignettes. Respondents correctly interpreted 81.8% HS-TM vs. 56.5% regression assessments, p < 0.001. Respondents “trusted the results” for 70.9% HS-TM vs. 58.2% regression assessments, p = 0.03. Nine concerns about credibility were identified: inadequate capture of case-mix and/or illness severity; inability to account for specialized programs (e.g., transplant center); comparison to geographically disparate hospitals; equating mortality with quality; lack of criterion standards; low power; comparison to dissimilar hospitals; generation of rankings; and lack of transparency. Five concerns were equally relevant to both methods, one more pertinent to HS-TM, and three more pertinent to regression. Assessments were mainly used to trigger further quality evaluation (a “check oil light”) and motivate behavior change. Conclusions HS-TM-based performance assessments were more interpretable and more credible to VA hospital leaders than regression-based assessments. However, leaders had a similar set of concerns related to credibility for both methods and felt both were best used as a screen for further evaluation.
Related Links	https://bmchealthservres.biomedcentral.com/counter/pdf/10.1186/s12913-022-08124-w.pdf
Ending Page	13
Page Count	13
Starting Page	1
File Format	HTM / HTML
ISSN	14726963
DOI	10.1186/s12913-022-08124-w
Journal	BMC Health Services Research
Issue Number	1
Volume Number	22
Language	English
Publisher	BioMed Central
Publisher Date	2022-06-03
Access Restriction	Open
Subject Keyword	Public Health Health Administration Health Informatics Nursing Research Quality of health care Benchmarking Risk adjustment Hospital mortality
Content Type	Text
Resource Type	Article
Subject	Health Policy

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

What is the empirical evidence that hospitals with higher-risk adjusted mortality rates provide poorer quality care? A systematic review of the literature

Risk adjustment methods for Home Care Quality Indicators (HCQIs) based on the minimum data set for home care

Risk-adjustment methods for all-payer comparative performance reporting in Vermont

COVID-19 inpatient care performance in the unified health system, São Paulo state, Brazil: an application of standardized mortality ratio for hospitals’ comparisons

An administrative model for benchmarking hospitals on their 30-day sepsis mortality

Hospital performance comparison of inpatient fall rates; the impact of risk adjusting for patient-related factors: a multicentre cross-sectional survey

Improving hospital quality risk-adjustment models using interactions identified by hierarchical group lasso regularisation

Assessment of hospital performance with a case-mix standardized mortality model using an existing administrative database in Japan

Hospital quality measures: are process indicators associated with hospital standardized mortality ratios in French acute care hospitals?

Interpretability, credibility, and usability of hospital-specific template matching versus regression-based hospital performance assessments; a multiple methods study

Similar Documents

What is the empirical evidence that hospitals with higher-risk adjusted mortality rates provide poorer quality care? A systematic review of the literature

Risk adjustment methods for Home Care Quality Indicators (HCQIs) based on the minimum data set for home care

Risk-adjustment methods for all-payer comparative performance reporting in Vermont

COVID-19 inpatient care performance in the unified health system, São Paulo state, Brazil: an application of standardized mortality ratio for hospitals’ comparisons

An administrative model for benchmarking hospitals on their 30-day sepsis mortality

Hospital performance comparison of inpatient fall rates; the impact of risk adjusting for patient-related factors: a multicentre cross-sectional survey

Improving hospital quality risk-adjustment models using interactions identified by hierarchical group lasso regularisation

Assessment of hospital performance with a case-mix standardized mortality model using an existing administrative database in Japan

Hospital quality measures: are process indicators associated with hospital standardized mortality ratios in French acute care hospitals?

Interpretability, credibility, and usability of hospital-specific template matching versus regression-based hospital performance assessments; a multiple methods study