The Hamlyn Centre for Robotic Surgery, Imperial College London, London, UK; Department of Computing, Imperial College London, London, UK.
The Hamlyn Centre for Robotic Surgery, Imperial College London, London, UK; Wellcome/EPSRC Centre for Interventional & Surgical Sciences (WEISS), University College London, London, UK; Centre for Medical Image Computing, University College London, London, UK; Department of Computer Science, University College London, London, UK; Department of Surgery and Cancer, Imperial College London, London, UK.
Med Image Anal. 2018 Aug;48:162-176. doi: 10.1016/j.media.2018.06.004. Epub 2018 Jun 15.
Surgical guidance and decision making could be improved with accurate and real-time measurement of intra-operative data including shape and spectral information of the tissue surface. In this work, a dual-modality endoscopic system has been proposed to enable tissue surface shape reconstruction and hyperspectral imaging (HSI). This system centers around a probe comprised of an incoherent fiber bundle, whose fiber arrangement is different at the two ends, and miniature imaging optics. For 3D reconstruction with structured light (SL), a light pattern formed of randomly distributed spots with different colors is projected onto the tissue surface, creating artificial texture. Pattern decoding with a Convolutional Neural Network (CNN) model and a customized feature descriptor enables real-time 3D surface reconstruction at approximately 12 frames per second (FPS). In HSI mode, spatially sparse hyperspectral signals from the tissue surface can be captured with a slit hyperspectral imager in a single snapshot. A CNN based super-resolution model, namely "super-spectral-resolution" network (SSRNet), has also been developed to estimate pixel-level dense hypercubes from the endoscope cameras standard RGB images and the sparse hyperspectral signals, at approximately 2 FPS. The probe, with a 2.1 mm diameter, enables the system to be used with endoscope working channels. Furthermore, since data acquisition in both modes can be accomplished in one snapshot, operation of this system in clinical applications is minimally affected by tissue surface movement and deformation. The whole apparatus has been validated on phantoms and tissue (ex vivo and in vivo), while initial measurements on patients during laryngeal surgery show its potential in real-world clinical applications.
手术指导和决策可以通过准确和实时地测量术中数据得到改善，包括组织表面的形状和光谱信息。在这项工作中，提出了一种双模内窥镜系统，以实现组织表面形状重建和高光谱成像（HSI）。该系统以一个由非相干光纤束组成的探头为中心，其光纤排列在两端不同，还有微型成像光学器件。对于结构光（SL）的 3D 重建，用不同颜色的随机分布点形成的光图案投射到组织表面，创建人工纹理。使用卷积神经网络（CNN）模型和定制的特征描述符进行图案解码，可实现每秒约 12 帧的实时 3D 表面重建。在 HSI 模式下，可在单个快照中用狭缝高光谱成像仪捕获来自组织表面的空间稀疏高光谱信号。还开发了基于 CNN 的超分辨率模型，即“超光谱分辨率”网络（SSRNet），可从内窥镜摄像机的标准 RGB 图像和稀疏高光谱信号中估计像素级密集超立方体，大约每秒 2 帧。直径为 2.1mm 的探头可使系统用于内窥镜工作通道。此外，由于两种模式的数据采集都可以在一个快照中完成，因此该系统在临床应用中的操作受组织表面运动和变形的影响最小。整个仪器已经在体模和组织（离体和体内）上进行了验证，而在喉科手术期间对患者的初步测量表明了它在实际临床应用中的潜力。
