School of Electronic and Information Engineering, Harbin Institute of Technology, Harbin 150001, China.
School of Electronic and Information Engineering, Harbin Institute of Technology, Harbin 150001, China.
Comput Med Imaging Graph. 2024 Jun;114:102367. doi: 10.1016/j.compmedimag.2024.102367. Epub 2024 Mar 16.
Whole Slide Imaging and Hyperspectral Microscopic Imaging provide great quality data with high spatial and spectral resolution for histopathology. Existing Hyperspectral Whole Slide Imaging systems combine the advantages of the techniques above, thus providing rich information for pathological diagnosis. However, it cannot avoid the problems of slow acquisition speed and mass data storage demand. Inspired by the spectral reconstruction task in computer vision and remote sensing, the Swin-Spectral Transformer U-Net (SSTU) has been developed to reconstruct Hyperspectral Whole Slide images (HWSis) from multiple Hyperspectral Microscopic images (HMis) of small Field of View and Whole Slide images (WSis). The Swin-Spectral Transformer (SST) module in SSTU takes full advantage of Transformer in extracting global attention. Firstly, Swin Transformer is exploited in space domain, which overcomes the high computation cost in Vision Transformer structures, while it maintains the spatial features extracted from WSis. Furthermore, Spectral Transformer is exploited to collect the long-range spectral features in HMis. Combined with the multi-scale encoder-bottleneck-decoder structure of U-Net, SSTU network is formed by sequential and symmetric residual connections of SSTs, which reconstructs a selected area of HWSi from coarse to fine. Qualitative and quantitative experiments prove the performance of SSTU in HWSi reconstruction task superior to other state-of-the-art spectral reconstruction methods.
全玻片成像和高光谱显微镜成像为组织病理学提供了具有高空间和光谱分辨率的高质量数据。现有的高光谱全玻片成像系统结合了上述技术的优点，从而为病理诊断提供了丰富的信息。然而，它不能避免采集速度慢和大量数据存储需求的问题。受计算机视觉和遥感中光谱重建任务的启发，开发了 Swin-Spectral Transformer U-Net（SSTU），以便从小视场的多个高光谱显微镜图像（HMis）和全玻片图像（WSis）中重建高光谱全玻片图像（HWSis）。SSTU 中的 Swin-Spectral Transformer（SST）模块充分利用了 Transformer 来提取全局注意力。首先，在空域中利用 Swin Transformer，克服了 Vision Transformer 结构中的高计算成本，同时保持了从 WSis 中提取的空间特征。此外，还利用 Spectral Transformer 来收集 HMis 中的长程光谱特征。结合 U-Net 的多尺度编码器-瓶颈-解码器结构，通过 SST 的顺序和对称残差连接形成 SSTU 网络，从粗到细重建 HWSi 的选定区域。定性和定量实验证明了 SSTU 在 HWSi 重建任务中的性能优于其他最先进的光谱重建方法。
