Alfasly Saghir, Alabtah Ghazal, Hemati Sobhan, Kalari Krishna Rani, Garcia Joaquin J, Tizhoosh H R
KIMIA Lab, Department of Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, USA.
Division of Computational Biology, Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, USA.
Sci Rep. 2025 Feb 1;15(1):3990. doi: 10.1038/s41598-025-88545-9.
We evaluated several foundation models in histopathology for image retrieval using a zero-shot approach. These models generated embeddings that were directly employed for retrieval without additional fine-tuning. Our experiments were conducted on diagnostic slides from The Cancer Genome Atlas (TCGA), which covers 23 organs and 117 cancer subtypes. We used Yottixel as the framework for whole-slide image (WSI) retrieval via patch-based embeddings. Retrieval performance was evaluated using macro-averaged F1 scores for top-1, top-3, and top-5 retrievals. The top-5 retrieval F1 scores indicated varying levels of performance: Yottixel-DenseNet (27% ± 13%), Yottixel-UNI (42% ± 14%), Yottixel-Virchow (40% ± 13%), Yottixel-GigaPath (41% ± 13%), and GigaPath WSI (40% ± 14%). These results demonstrate the potential and limitations of foundation models for histopathology image retrieval, underscoring the need for further advancements in embedding and retrieval techniques.
我们使用零样本方法评估了组织病理学中用于图像检索的几种基础模型。这些模型生成的嵌入向量可直接用于检索，无需额外的微调。我们的实验是在来自癌症基因组图谱（TCGA）的诊断切片上进行的，该图谱涵盖23个器官和117种癌症亚型。我们使用Yottixel作为通过基于补丁的嵌入进行全切片图像（WSI）检索的框架。使用前1、前3和前5检索的宏平均F1分数评估检索性能。前5检索的F1分数表明了不同的性能水平：Yottixel-DenseNet（27%±13%）、Yottixel-UNI（42%±14%）、Yottixel-Virchow（40%±13%）、Yottixel-GigaPath（41%±13%）和GigaPath WSI（40%±14%）。这些结果证明了基础模型在组织病理学图像检索中的潜力和局限性，强调了在嵌入和检索技术方面进一步改进的必要性。
