Czempiel Tobias, Roddan Alfie, Leiloglou Maria, Hu Zepeng, O'Neill Kevin, Anichini Giulio, Stoyanov Danail, Elson Daniel
EnAcuity Limited London UK.
Hamlyn Centre for Robotic Surgery Department of Surgery and Cancer Imperial College London London UK.
Healthc Technol Lett. 2024 Nov 25;11(6):307-317. doi: 10.1049/htl2.12098. eCollection 2024 Dec.
This study investigates the reconstruction of hyperspectral signatures from RGB data to enhance surgical imaging, utilizing the publicly available HeiPorSPECTRAL dataset from porcine surgery and an in-house neurosurgery dataset. Various architectures based on convolutional neural networks (CNNs) and transformer models are evaluated using comprehensive metrics. Transformer models exhibit superior performance in terms of RMSE, SAM, PSNR and SSIM by effectively integrating spatial information to predict accurate spectral profiles, encompassing both visible and extended spectral ranges. Qualitative assessments demonstrate the capability to predict spectral profiles critical for informed surgical decision-making during procedures. Challenges associated with capturing both the visible and extended hyperspectral ranges are highlighted using the MAE, emphasizing the complexities involved. The findings open up the new research direction of hyperspectral reconstruction for surgical applications and clinical use cases in real-time surgical environments.
本研究利用公开可用的猪手术HeiPorSPECTRAL数据集和内部神经外科数据集，研究从RGB数据重建高光谱特征以增强手术成像。使用综合指标评估了基于卷积神经网络（CNN）和Transformer模型的各种架构。Transformer模型通过有效整合空间信息以预测准确的光谱轮廓（包括可见光谱范围和扩展光谱范围），在均方根误差（RMSE）、光谱角映射（SAM）、峰值信噪比（PSNR）和结构相似性（SSIM）方面表现出卓越性能。定性评估表明，该模型有能力预测手术过程中对明智手术决策至关重要的光谱轮廓。平均绝对误差（MAE）突出了在捕获可见光谱范围和扩展高光谱范围时面临的挑战，强调了其中涉及的复杂性。这些发现为手术应用以及实时手术环境中的临床用例开辟了高光谱重建的新研究方向。
