Liu Yuxuan, Zhou Jiasheng, Luo Yating, Chen Sung-Liang, Guo Yao, Yang Guang-Zhong
Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China; Shanghai Key Laboratory of Flexible Medical Robotics, Tongren Hospital, Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, 200240, China.
Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China.
Med Image Anal. 2025 Oct;105:103698. doi: 10.1016/j.media.2025.103698. Epub 2025 Jun 30.
Robot-assisted microsurgery is a promising technique for a number of clinical specialties including neurosurgery. One of the prerequisites of such procedures is accurate vision guidance, delineating not only the exposed surface details but also embedded microvasculature. Conventional microscopic cameras used for vascular imaging are susceptible to specular reflections and changes in ambient light with low tissue resolution and contrast. Photoacoustic microscopy (PAM) is emerging as a promising tool and increasingly used for vascular imaging due to its high image resolution and tissue contrast. This paper presents a fused microscopic imaging scheme that integrates standard surgical microscopy with PAM for improved intraoperative visualization and guidance. We propose the FPM-RNet to Fuse Photoacoustic and surgical Microscopic imaging via cross-modality Representation and Registration Network. A MOdality Representation Network (MORNet) is used to extract unified feature representation across white-light and PAM modalities, and a Hierarchical Iterative Registration Network (HIRNet) is used to establish the correspondence between the two modalities in a coarse-to-fine manner based on multi-resolution feature maps. A synthetic dataset with ground truth correspondence and an in vivo dataset of mouse brain vasculature are used to evaluate our proposed network. Extensive validation on the two datasets has shown significant improvements compared to the current state-of-the-art methods assessed with intersection over union and Dice scores (10.3% and 6.6% on the synthetic dataset and 15.9% and 11.8% on the in vivo dataset, respectively).
机器人辅助显微手术对于包括神经外科在内的许多临床专科来说是一项很有前景的技术。此类手术的前提条件之一是精确的视觉引导，不仅要描绘出暴露的表面细节，还要勾勒出内部的微血管。用于血管成像的传统显微镜相机容易受到镜面反射和环境光变化的影响，组织分辨率和对比度较低。光声显微镜（PAM）作为一种很有前景的工具正在兴起，并因其高图像分辨率和组织对比度而越来越多地用于血管成像。本文提出了一种融合显微成像方案，将标准手术显微镜与PAM相结合，以改善术中可视化和引导。我们提出了FPM-RNet，通过跨模态表示和配准网络融合光声和手术显微成像。模态表示网络（MORNet）用于跨白光和PAM模态提取统一的特征表示，分层迭代配准网络（HIRNet）用于基于多分辨率特征图以粗到细的方式建立两种模态之间的对应关系。使用具有真实对应关系的合成数据集和小鼠脑血管的体内数据集来评估我们提出的网络。与使用交并比和Dice分数评估的当前最先进方法相比，在这两个数据集上的广泛验证显示出显著改进（在合成数据集上分别提高了10.3%和6.6%，在体内数据集上分别提高了15.9%和11.8%）。
