Faculty, Staff and Student Publications

Publication Date

4-30-2025

Journal

Clinical Infectious Disease

Abstract

Background: Diagnostic test evaluation requires a reference standard. We describe an approach for creating a reference standard for acute infection using unrestricted adjudication and apply it to compare biomarker tools.

Methods: Adults and children with suspected acute infection enrolled in three prospective studies at emergency departments and urgent cares were included. Adjudicators, blinded to C-reactive protein, procalcitonin, and MeMed BV (MMBV), labeled each case (bacterial/viral/non-infectious/indeterminate). Initial adjudication involved 3 adjudicators. Reference standard cohorts were defined: Microbiologically confirmed (3/3 adjudicators concur with high confidence and a concordant microbiological finding), unanimous (3/3 adjudicators concur with high confidence), suspected (3/3 adjudicators concur with high/moderate confidence or 2/3 adjudicators concur with high confidence), and all-inclusive (remaining unlabeled cases were reviewed by up to 7 additional adjudicators until reaching a leading label).

Results: Among 1016 patients, 156 difficult-to-diagnose cases required over 3 adjudicators. The area under the receiver operating characteristic curve in the microbiologically confirmed (n = 427), unanimous (n = 565), suspected (n = 860), and all-inclusive (n = 1016) cohorts for MMBV were 0.98 (95% confidence interval .94-1.00), 0.98 (.95-1.00), 0.95 (.92-.98) and 0.90 (.87-.93), respectively, and for procalcitonin were 0.69 (.57-.81), 0.77 (.68-.86), 0.74 (.68-.80) and 0.70 (.65-.75), respectively. A delta in performance between MMBV and procalcitonin was maintained across the different cohorts.

Conclusions: Creating a reference standard that includes difficult-to-diagnose cases demands an approach to addressing diagnostic uncertainty in acute infections. Tool performance depends on the reference standard applied and decreases as the difficulty to diagnose increases, highlighting the importance of using the same reference standard when comparing tools.

Keywords

Humans, Virus Diseases, Bacterial Infections, Adult, Male, Female, Prospective Studies, Child, Biomarkers, Reference Standards, Middle Aged, Procalcitonin, Child, Preschool, Aged, Adolescent, C-Reactive Protein, Young Adult, Diagnostic Tests, Routine, Infant, Sensitivity and Specificity, Emergency Service, Hospital

DOI

10.1093/cid/ciae656

PMID

39750735

PMCID

PMC12043069

PubMedCentral® Posted Date

1-3-2025

PubMedCentral® Full Text Version

Post-print

Published Open-Access

yes

Share

COinS
 
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.