Translational Projects (Open Access)
Graduation Date
Spring 2022
Degree Name
Doctorate in Health Informatics (DHI)
School Name
The University of Texas School of Biomedical Informatics at Houston
Advisory Committee
Robert E. Murphy, MD
Abstract
Background: Medical diagnosis may be the most complex task attempted by humans. Studies estimate that 95% of diagnoses in outpatient care are accurate, implying that the annual rate of inaccurate diagnoses is 12 million in the US alone, with the potential for patient harm in about half. A well-researched differential might reduce inaccurate diagnoses by offering alternatives matching the patient’s symptoms. This study searched the literature for articles evaluating the diagnostic performance of commercially available computerized diagnostic decision support systems. This search led to selecting Isabel Pro, developed by Isabel Healthcare, Ltd. of Haslemere, UK.
Evaluation and Validation: A computerized diagnostic decision support system should respond adequately to four questions: What is the “diagnostic retrieval accuracy”? Does it perform as well as clinicians? When provided with the differential, do clinicians improve diagnostic accuracy? Is it easily incorporated into routine practice? The project validated the diagnostic retrieval accuracy of Isabel Pro using 46 cases with a previously confirmed diagnosis. The confirmed diagnosis appeared in Isabel Pro’s differential in 24 cases (52.2%), outperforming even internal medicine faculty (47%). Using those 24 cases and the differentials produced, the author conducted a diagnostic challenge that involved 120 McGovern Medical School residents. The residents produced 406 diagnoses, of which 105 (25.9%) were correct without the differentials, and 37 were correct post-consultation, a 9.1% absolute improvement. In responses, 75.1% of the participants agreed the differentials would be helpful in routine practice, and 64.1% agreed they would consult the differentials if available.
Implementation: The project successfully proposed Isabel Pro as a solution to UT practice leadership on September 16, 2021, and incorporated the system into the Epic EHR as a menu line link on November 30, 2021. This system-wide integration also included a QR code for downloading Isabel Pro to a mobile device. Usage of Isabel Pro in the practices of UTPhysicians began on December 8, 2021. Results: The project concluded data collection after 86 days on March 4, 2022, with usage showing a steady increase in the final three weeks. The project produced 73 unique users (37 faculty and 36 residents). The user survey responses showed 83.3% agreeing they would consult the differential generated by Isabel Pro if available at every patient encounter (+19.2% compared to the challenge survey) and 77.8% agreeing that the suggestions would be helpful in routine practice (+2.7% compared to the challenge survey). More than one-third (36.8%) responded that they changed their diagnosis in response to the differential. Limitations: Only usage statistics were analyzed; the system records no reason for the clinician discontinuing a diagnostic session. Only 20 participants responded out of 73 (27.4%), so even though the respondents represented a spread of experience levels, the results may not represent the total number of potential users. The project covered a limited period of 86 days.
Conclusions: Diagnostic inaccuracy is a significant patient safety concern. Studies show that computerized diagnostic decision support systems improve diagnostic accuracy, but they are not wide implementation lags despite these findings. This project demonstrated the feasibility of implementing such a well-known system in academic medical practice. The responses to the surveys demonstrate favorable opinions about the system’s perceived usefulness. Active communication and dissemination programs may be essential to improve and sustain use.
Recommended Citation
Bridges, Joe, "Evaluation, Validation & Implementation of a Computerized Diagnostic Decision Support System in Primary Practice" (2022). Translational Projects (Open Access). 1.
https://digitalcommons.library.tmc.edu/uthsbmi_dhi_dissertations/1
Keywords
Computerized diagnostic decision support, diagnostic accuracy, patient safety, differential diagnosis