Li Zheyu, Li Bin, Eliceiri Kevin W, Narayanan Vijaykrishnan
Department of Computer Science and Engineering, Pennsylvania State University, State College, PA 16801, USA.
Authors contributed equally.
Biomed Opt Express. 2023 Jan 6;14(2):667-686. doi: 10.1364/BOE.477515. eCollection 2023 Feb 1.
Whole slide image (WSI) analysis is increasingly being adopted as an important tool in modern pathology. Recent deep learning-based methods have achieved state-of-the-art performance on WSI analysis tasks such as WSI classification, segmentation, and retrieval. However, WSI analysis requires a significant amount of computation resources and computation time due to the large dimensions of WSIs. Most of the existing analysis approaches require the complete decompression of the whole image exhaustively, which limits the practical usage of these methods, especially for deep learning-based workflows. In this paper, we present compression domain processing-based computation efficient analysis workflows for WSIs classification that can be applied to state-of-the-art WSI classification models. The approaches leverage the pyramidal magnification structure of WSI files and compression domain features that are available from the raw code stream. The methods assign different decompression depths to the patches of WSIs based on the features directly retained from compressed patches or partially decompressed patches. Patches from the low-magnification level are screened by attention-based clustering, resulting in different decompression depths assigned to the high-magnification level patches at different locations. A finer-grained selection based on compression domain features from the file code stream is applied to select further a subset of the high-magnification patches that undergo a full decompression. The resulting patches are fed to the downstream attention network for final classification. Computation efficiency is achieved by reducing unnecessary access to the high zoom level and expensive full decompression. With the number of decompressed patches reduced, the time and memory costs of downstream training and inference procedures are also significantly reduced. Our approach achieves a 7.2× overall speedup, and the memory cost is reduced by 1.1 orders of magnitudes, while the resulting model accuracy is comparable to the original workflow.
全玻片图像（WSI）分析在现代病理学中越来越被视为一种重要工具。最近基于深度学习的方法在WSI分析任务（如WSI分类、分割和检索）中取得了领先的性能。然而，由于WSI尺寸巨大，WSI分析需要大量的计算资源和计算时间。现有的大多数分析方法都需要对整个图像进行彻底的完全解压缩，这限制了这些方法的实际应用，特别是对于基于深度学习的工作流程。在本文中，我们提出了基于压缩域处理的计算高效的WSI分类分析工作流程，该流程可应用于先进的WSI分类模型。这些方法利用了WSI文件的金字塔放大结构和从原始代码流中获取的压缩域特征。基于直接从压缩补丁或部分解压缩补丁中保留的特征，为WSI的补丁分配不同的解压缩深度。低倍率级别的补丁通过基于注意力的聚类进行筛选，从而为不同位置的高倍率级别补丁分配不同的解压缩深度。基于文件代码流中的压缩域特征进行更细粒度的选择，以进一步选择进行完全解压缩的高倍率补丁子集。将得到的补丁输入到下游注意力网络进行最终分类。通过减少对高缩放级别不必要的访问和昂贵的完全解压缩来实现计算效率。随着解压缩补丁数量的减少，下游训练和推理过程的时间和内存成本也显著降低。我们的方法实现了7.2倍的整体加速，内存成本降低了1.1个数量级，而得到的模型精度与原始工作流程相当。
