A Visual-Omics Foundation Model To Bridge Histopathology With Spatial Transcriptomics

Authors

Weiqing Chen
Pengzhi Zhang
Tu N Tran
Yiwei Xiao
Shengyu Li
Vrutant V Shah
Hao Cheng
Kristopher W Brannan
Keith Youker
Li Lai
Longhou Fang
Yu Yang
Nhat-Tu Le
Jun-Ichi Abe
Shu-Hsia Chen
Qin Ma
Ken Chen
Qianqian Song
John P Cooke
Guangyu Wang

Publication Date

7-1-2025

Journal

Nature Methods

DOI

10.1038/s41592-025-02707-1

PMID

40442373

PMCID

PMC12240810

PubMedCentral® Posted Date

5-29-2025

PubMedCentral® Full Text Version

Post-print

Abstract

Artificial intelligence has revolutionized computational biology. Recent developments in omics technologies, including single-cell RNA sequencing and spatial transcriptomics, provide detailed genomic data alongside tissue histology. However, current computational models focus on either omics or image analysis, lacking their integration. To address this, we developed OmiCLIP, a visual-omics foundation model linking hematoxylin and eosin images and transcriptomics using tissue patches from Visium data. We transformed transcriptomic data into 'sentences' by concatenating top-expressed gene symbols from each patch. We curated a dataset of 2.2 million paired tissue images and transcriptomic data across 32 organs to train OmiCLIP integrating histology and transcriptomics. Building on OmiCLIP, our Loki platform offers five key functions: tissue alignment, annotation via bulk RNA sequencing or marker genes, cell-type decomposition, image-transcriptomics retrieval and spatial transcriptomics gene expression prediction from hematoxylin and eosin-stained images. Compared with 22 state-of-the-art models on 5 simulations, and 19 public and 4 in-house experimental datasets, Loki demonstrated consistent accuracy and robustness.

Keywords

Humans, Transcriptome, Computational Biology, Gene Expression Profiling, Genomics, Image Processing, Computer-Assisted, Artificial Intelligence, Single-Cell Analysis, Computational models, Software, Machine learning, Computational platforms and environments

Published Open-Access

yes

Recommended Citation

Chen, Weiqing; Zhang, Pengzhi; Tran, Tu N; et al., "A Visual-Omics Foundation Model To Bridge Histopathology With Spatial Transcriptomics" (2025). Faculty, Staff and Student Publications. 4199.
https://digitalcommons.library.tmc.edu/uthgsbs_docs/4199