Stanford University, Department of Otolaryngology - Head and Neck Surgery, Palo Alto, California, United States.
Stanford University, Department of Electrical Engineering, Stanford, California, United States.
J Biomed Opt. 2023 Jan;28(1):016004. doi: 10.1117/1.JBO.28.1.016004. Epub 2023 Jan 28.
Accurate identification of tissues is critical for performing safe surgery. Combining multispectral imaging (MSI) with deep learning is a promising approach to increasing tissue discrimination and classification. Evaluating the contributions of spectral channels to tissue discrimination is important for improving MSI systems.
Develop a metric to quantify the contributions of individual spectral channels to tissue classification in MSI.
MSI was integrated into a digital operating microscope with three sensors and seven illuminants. Two convolutional neural network (CNN) models were trained to classify 11 head and neck tissue types using white light (RGB) or MSI images. The signal to noise ratio (SNR) of spectral channels was compared with the impact of channels on tissue classification performance as determined using CNN visualization methods.
Overall tissue classification accuracy was higher with use of MSI images compared with RGB images, both for classification of all 11 tissue types and binary classification of nerve and parotid ( ). Removing spectral channels with reduced tissue classification accuracy.
The spectral channel SNR is a useful metric for both understanding CNN tissue classification and quantifying the contributions of different spectral channels in an MSI system.
准确识别组织对于进行安全手术至关重要。将多光谱成像 (MSI) 与深度学习相结合是提高组织辨别和分类的有前途的方法。评估光谱通道对组织辨别能力的贡献对于改进 MSI 系统很重要。
开发一种度量标准，以量化 MSI 中各个光谱通道对组织分类的贡献。
MSI 被集成到具有三个传感器和七个照明光源的数字手术显微镜中。使用卷积神经网络 (CNN) 模型对 11 种头颈部组织类型进行分类，分别使用白光 (RGB) 或 MSI 图像。将光谱通道的信噪比 (SNR) 与使用 CNN 可视化方法确定的通道对组织分类性能的影响进行比较。
与使用 RGB 图像相比，使用 MSI 图像进行整体组织分类准确性更高，包括对所有 11 种组织类型的分类和神经与腮腺的二进制分类 ( )。减少光谱通道会降低组织分类准确性。
光谱通道 SNR 是理解 CNN 组织分类和量化 MSI 系统中不同光谱通道贡献的有用指标。
