Faculty, Staff and Student Publications
Publication Date
9-1-2022
Journal
Journal of Applied Clinical Medical Physics
Abstract
Purpose: To develop a checklist that improves the rate of error detection during the plan review of automatically generated radiotherapy plans.
Methods: A custom checklist was developed using guidance from American Association of Physicists in Medicine task groups 275 and 315 and the results of a failure modes and effects analysis of the Radiation Planning Assistant (RPA), an automated contouring and treatment planning tool. The preliminary checklist contained 90 review items for each automatically generated plan. In the first study, eight physicists were recruited from our institution who were familiar with the RPA. Each physicist reviewed 10 artificial intelligence-generated resident treatment plans from the RPA for safety and plan quality, five of which contained errors. Physicists performed plan checks, recorded errors, and rated each plan's clinical acceptability. Following a 2-week break, physicists reviewed 10 additional plans with a similar distribution of errors using our customized checklist. Participants then provided feedback on the usability of the checklist and it was modified accordingly. In a second study, this process was repeated with 14 senior medical physics residents who were randomly assigned to checklist or no checklist for their reviews. Each reviewed 10 plans, five of which contained errors, and completed the corresponding survey.
Results: In the first study, the checklist significantly improved the rate of error detection from 3.4 ± 1.1 to 4.4 ± 0.74 errors per participant without and with the checklist, respectively (p = 0.02). Error detection increased by 20% when the custom checklist was utilized. In the second study, 2.9 ± 0.84 and 3.5 ± 0.84 errors per participant were detected without and with the revised checklist, respectively (p = 0.08). Despite the lack of statistical significance for this cohort, error detection increased by 18% when the checklist was utilized.
Conclusion: Our results indicate that the use of a customized checklist when reviewing automated treatment plans will result in improved patient safety.
Keywords
Artificial Intelligence, Checklist, Humans, Radiotherapy Dosage, Radiotherapy Planning, Computer-Assisted, Radiotherapy, Intensity-Modulated, automation, checklist, patient safety, quality assurance, risk assessment
DOI
10.1002/acm2.13694
PMID
35775105
PMCID
PMC9512344
PubMedCentral® Posted Date
6-30-2022
PubMedCentral® Full Text Version
Post-print
Published Open-Access
yes
Included in
Bioinformatics Commons, Biomedical Informatics Commons, Genetic Phenomena Commons, Health and Medical Physics Commons, Medical Genetics Commons, Oncology Commons
