Faculty, Staff and Student Publications
Publication Date
4-30-2025
Journal
Clinical Infectious Disease
Abstract
Background: Diagnostic test evaluation requires a reference standard. We describe an approach for creating a reference standard for acute infection using unrestricted adjudication and apply it to compare biomarker tools.
Methods: Adults and children with suspected acute infection enrolled in three prospective studies at emergency departments and urgent cares were included. Adjudicators, blinded to C-reactive protein, procalcitonin, and MeMed BV (MMBV), labeled each case (bacterial/viral/non-infectious/indeterminate). Initial adjudication involved 3 adjudicators. Reference standard cohorts were defined: Microbiologically confirmed (3/3 adjudicators concur with high confidence and a concordant microbiological finding), unanimous (3/3 adjudicators concur with high confidence), suspected (3/3 adjudicators concur with high/moderate confidence or 2/3 adjudicators concur with high confidence), and all-inclusive (remaining unlabeled cases were reviewed by up to 7 additional adjudicators until reaching a leading label).
Results: Among 1016 patients, 156 difficult-to-diagnose cases required over 3 adjudicators. The area under the receiver operating characteristic curve in the microbiologically confirmed (n = 427), unanimous (n = 565), suspected (n = 860), and all-inclusive (n = 1016) cohorts for MMBV were 0.98 (95% confidence interval .94-1.00), 0.98 (.95-1.00), 0.95 (.92-.98) and 0.90 (.87-.93), respectively, and for procalcitonin were 0.69 (.57-.81), 0.77 (.68-.86), 0.74 (.68-.80) and 0.70 (.65-.75), respectively. A delta in performance between MMBV and procalcitonin was maintained across the different cohorts.
Conclusions: Creating a reference standard that includes difficult-to-diagnose cases demands an approach to addressing diagnostic uncertainty in acute infections. Tool performance depends on the reference standard applied and decreases as the difficulty to diagnose increases, highlighting the importance of using the same reference standard when comparing tools.
Keywords
Humans, Virus Diseases, Bacterial Infections, Adult, Male, Female, Prospective Studies, Child, Biomarkers, Reference Standards, Middle Aged, Procalcitonin, Child, Preschool, Aged, Adolescent, C-Reactive Protein, Young Adult, Diagnostic Tests, Routine, Infant, Sensitivity and Specificity, Emergency Service, Hospital
DOI
10.1093/cid/ciae656
PMID
39750735
PMCID
PMC12043069
PubMedCentral® Posted Date
1-3-2025
PubMedCentral® Full Text Version
Post-print
Published Open-Access
yes