Assessment of Racial Bias within the Risk Analysis Index of Frailty

Authors

Michael A Jacobs
Leslie R M Hausmann
Robert M Handzel
Susanne Schmidt
Carly A Jacobs
Daniel E Hall

Language

English

Publication Date

12-1-2024

Journal

Annals of Surgery Open

DOI

10.1097/AS9.0000000000000490

PMID

39711679

PMCID

PMC11661760

PubMedCentral® Posted Date

9-25-2024

PubMedCentral® Full Text Version

Post-print

Abstract

Objective: Our objective was to assess potential racial bias within the Risk Analysis Index (RAI).

Background: Patient risk measures are rarely tested for racial bias. Measures of frailty, like the RAI, need to be evaluated for poor predictive performance among Black patients.

Methods: Retrospective cohort study using April 2010-March 2019 Veterans Affairs Surgical Quality Improvement Program and 2010-2019 National Surgical Quality Improvement Program data. The performance of the RAI and several potential variants were compared between Black and White cases using various metrics to predict mortality (180-day for Veterans Affairs Surgical Quality Improvement Program, 30-day for National Surgical Quality Improvement Program).

Results: Using the current, clinical threshold, the RAI performed as good or better among Black cases across various performance metrics versus White. When a higher threshold was used, Black cases had higher true positive rates but lower true negative rates, yielding 2.0% higher balanced accuracy. No RAI variant noticeably eliminated bias, improved parity across both true positives and true negatives, or improved overall model performance.

Conclusions: The RAI tends to predict mortality among Black patients better than it predicts mortality among White patients. As existing bias-reducing techniques were not effective, further research into bias-reducing techniques is needed, especially for clinical risk predictions. We recommend using the RAI for both statistical analysis of surgical cohorts and quality improvement programs, such as the Surgical Pause.

Comments

This article has been corrected. See Ann Surg Open. 2024 Dec 24;6(1):e515.

Published Open-Access

yes

Recommended Citation

Jacobs, Michael A; Hausmann, Leslie R M; Handzel, Robert M; et al., "Assessment of Racial Bias within the Risk Analysis Index of Frailty" (2024). Faculty, Staff and Student Publications. 5895.
https://digitalcommons.library.tmc.edu/uthgsbs_docs/5895