Deep Learning Single-Cell Analysis for Cytologic Evaluation of Oral Potentially Malignant Disorders

Authors

• Michael P McRae
• Kritika S Rajsri
• Nadarajah Vigneswaran
• A Ross Kerr
• Spencer W Redding
• Martin H Thornhill
• Craig Murdoch
• Paul M Speight
• Nancy Ruel
• Rachelle Wolk
• Ryan R Ruff
• John T McDevitt

Language

English

Publication Date

7-13-2026

Journal

Scientific Reports

DOI

10.1038/s41598-026-47538-y

PMID

42437755

PMCID

PMC13357536

PubMedCentral® Posted Date

7-13-2026

PubMedCentral® Full Text Version

Post-print

Abstract

Oral potentially malignant disorders (OPMDs) such as leukoplakia and erythroplakia may harbor dysplasia or progress to oral squamous cell carcinoma (OSCC), yet visual inspection alone is unreliable for risk assessment. Cytology offers a minimally invasive adjunct, but conventional approaches depend on manual feature extraction and subjective review. Herein we report a deep learning (DL) object detection model that directly classifies four cell phenotypes: differentiated squamous epithelial (DSE) cells, small round (SR) cells, leukocytes, and lone nuclei. The DL model produced cytology-derived parameters that correlated strongly with histopathologic diagnoses across 692 subjects with OPMDs, OSCC, and healthy controls, including declining DSE cell proportion and increasing SR cells and leukocytes with disease severity (p <  0.0001). The oral cancer numerical index (OCNI) achieved AUROC values up to 0.99 for malignant versus healthy lesions, with excellent reliability (intra-class correlation coefficient ≥ 0.96). This reproducible, minimally invasive test provides a robust platform for early detection and surveillance of OPMDs.

Keywords

Humans, Deep Learning, Mouth Neoplasms, Single-Cell Analysis, Carcinoma, Squamous Cell, Precancerous Conditions, Oral potentially malignant disorders, Oral epithelial dysplasia, Oral squamous cell carcinoma, Deep learning, Artificial intelligence, Intelligent cytology microfluidics

Published Open-Access

yes

Recommended Citation

McRae, Michael P; Rajsri, Kritika S; Vigneswaran, Nadarajah; et al., "Deep Learning Single-Cell Analysis for Cytologic Evaluation of Oral Potentially Malignant Disorders" (2026). Faculty, Staff and Student Publications. 209.
https://digitalcommons.library.tmc.edu/uthdb_docs/209