Faculty, Staff and Student Publications
Publication Date
8-31-2025
Journal
Briefings in Bioinformatics
DOI
10.1093/bib/bbaf426
PMID
40889115
PMCID
PMC12400816
PubMedCentral® Posted Date
9-1-2025
PubMedCentral® Full Text Version
Post-print
Abstract
In silico perturbation models, computational methods that can predict cellular responses to perturbations, present an opportunity to reduce the need for costly and time-intensive in vitro experiments. Many recently proposed models predict high-dimensional cellular responses, such as gene or protein expression to perturbations such as gene knockout or drugs. However, evaluating in silico performance has largely relied on metrics such as $R^{2}$, which assess overall prediction accuracy but fail to capture biologically significant outcomes like the identification of differentially expressed (DE) genes. In this study, we present a novel evaluation framework that introduces the AUPRC metric to assess the precision and recall of DE gene predictions. By applying this framework to both single-cell and pseudo-bulked datasets, we systematically benchmark simple and advanced computational models. Our results highlight a significant discrepancy between $R^{2}$ and AUPRC, with models achieving high $R^{2}$ values but struggling to identify DE genes, as reflected in their low AUPRC values. This finding underscores the limitations of traditional evaluation metrics and the importance of biologically relevant assessments. Our framework provides a more comprehensive understanding of model capabilities, advancing the application of computational approaches in cellular perturbation research.
Keywords
Computer Simulation, Computational Biology, Humans, Algorithms, Gene Expression Profiling, Gene Expression Regulation
Published Open-Access
yes
Recommended Citation
Zhu, Hongxu; Asiaee, Amir; Azinfar, Leila; et al., "AUPRC: A Metric for Evaluating the Performance of In-Silico Perturbation Methods in Identifying Differentially Expressed Genes" (2025). Faculty, Staff and Student Publications. 4776.
https://digitalcommons.library.tmc.edu/uthgsbs_docs/4776
Included in
Bioinformatics Commons, Biomedical Informatics Commons, Genetic Phenomena Commons, Medical Genetics Commons, Oncology Commons