Roddan Alfie, Czempiel Tobias, Xu Chi, Xu Haozheng, Weld Alistair, Chalau Vadzim, Anichini Giulio, Elson Daniel S, Giannarou Stamatia
The Hamlyn Centre for Robotic Surgery, Imperial College London, Exhibition Rd, London, SW7 2AZ, UK.
Int J Comput Assist Radiol Surg. 2025 Apr 3. doi: 10.1007/s11548-025-03340-1.
This work presents a novel multimodal imaging platform that integrates hyperspectral imaging (HSI) and probe-based confocal laser endomicroscopy (pCLE) for improved brain tumor identification during neurosurgery. By combining these two modalities, we aim to enhance surgical navigation, addressing the limitations of using each modality when used independently.
We developed a multimodal imaging platform that integrates HSI and pCLE within an operating microscope setup using computer vision techniques. The system combines real-time, high-resolution HSI for macroscopic tissue analysis with pCLE for cellular-level imaging. The predictions of each modality made using Machine Learning methods are combined to improve tumor identification.
Our evaluation of the multimodal system revealed low spatial error, with minimal reprojection discrepancies, ensuring precise alignment between the HSI and pCLE. This combined imaging approach together with our multimodal tissue characterization algorithm significantly improves tumor identification, yielding higher Dice and Recall scores compared to using HSI or pCLE individually.
Our multimodal imaging platform represents a crucial first step toward enhancing tumor identification by combining HSI and pCLE modalities for the first time. We highlight improvements in metrics such as the Dice score and Recall, underscoring the potential for further advancements in this area.
本研究提出了一种新型多模态成像平台，该平台集成了高光谱成像（HSI）和基于探头的共聚焦激光内镜显微镜（pCLE），以在神经外科手术中改善脑肿瘤识别。通过结合这两种模式，我们旨在增强手术导航，解决单独使用每种模式时的局限性。
我们开发了一种多模态成像平台，该平台利用计算机视觉技术在手术显微镜设置中集成了HSI和pCLE。该系统将用于宏观组织分析的实时高分辨率HSI与用于细胞水平成像的pCLE相结合。使用机器学习方法对每种模式的预测进行组合，以改善肿瘤识别。
我们对多模态系统的评估显示空间误差低，重投影差异最小，确保了HSI和pCLE之间的精确对齐。这种联合成像方法与我们的多模态组织表征算法一起显著改善了肿瘤识别，与单独使用HSI或pCLE相比，产生了更高的Dice和召回分数。
我们的多模态成像平台首次将HSI和pCLE模式相结合，是朝着增强肿瘤识别迈出的关键第一步。我们强调了Dice分数和召回率等指标的改进，突出了该领域进一步发展的潜力。
