Ye Run Zhou, Iezzi Raymond
Department of Ophthalmology, Mayo Clinic, Rochester, MN 55905, USA.
J Pers Med. 2025 Jan 6;15(1):20. doi: 10.3390/jpm15010020.
: Augmented reality (AR) may allow vitreoretinal surgeons to leverage microscope-integrated digital imaging systems to analyze and highlight key retinal anatomic features in real time, possibly improving safety and precision during surgery. By employing convolutional neural networks (CNNs) for retina vessel segmentation, a retinal coordinate system can be created that allows pre-operative images of capillary non-perfusion or retinal breaks to be digitally aligned and overlayed upon the surgical field in real time. Such technology may be useful in assuring thorough laser treatment of capillary non-perfusion or in using pre-operative optical coherence tomography (OCT) to guide macular surgery when microscope-integrated OCT (MIOCT) is not available. : This study is a retrospective analysis involving the development and testing of a novel image-registration algorithm for vitreoretinal surgery. Fifteen anonymized cases of pars plana vitrectomy with epiretinal membrane peeling, along with corresponding preoperative fundus photographs and optical coherence tomography (OCT) images, were retrospectively collected from the Mayo Clinic database. We developed a TPU (Tensor-Processing Unit)-accelerated CNN for semantic segmentation of retinal vessels from fundus photographs and subsequent real-time image registration in surgical video streams. An iterative patch-wise cross-correlation (IPCC) algorithm was developed for image registration, with a focus on optimizing processing speeds and maintaining high spatial accuracy. The primary outcomes measured were processing speed in frames per second (FPS) and the spatial accuracy of image registration, quantified by the Dice coefficient between registered and manually aligned images. : When deployed on an Edge TPU, the CNN model combined with our image-registration algorithm processed video streams at a rate of 14 FPS, which is superior to processing rates achieved on other standard hardware configurations. The IPCC algorithm efficiently aligned pre-operative and intraoperative images, showing high accuracy in comparison to manual registration. : This study demonstrates the feasibility of using TPU-accelerated CNNs for enhanced AR in vitreoretinal surgery.
增强现实（AR）或许能让玻璃体视网膜外科医生利用集成于显微镜的数字成像系统，实时分析并突出显示关键的视网膜解剖特征，这可能会提高手术过程中的安全性和精准度。通过运用卷积神经网络（CNN）进行视网膜血管分割，能够创建一个视网膜坐标系，从而使术前毛细血管无灌注或视网膜裂孔的图像得以数字对齐，并实时叠加到手术视野上。当无法使用集成于显微镜的光学相干断层扫描（OCT）（MIOCT）时，此类技术在确保对毛细血管无灌注进行彻底激光治疗或利用术前OCT指导黄斑手术方面可能会有所帮助。
本研究是一项回顾性分析，涉及一种用于玻璃体视网膜手术的新型图像配准算法的开发与测试。从梅奥诊所数据库中回顾性收集了15例进行视网膜前膜剥离的扁平部玻璃体切除术的匿名病例，以及相应的术前眼底照片和光学相干断层扫描（OCT）图像。我们开发了一种由张量处理单元（TPU）加速的CNN，用于从眼底照片中对视网膜血管进行语义分割，并在手术视频流中进行后续的实时图像配准。还开发了一种迭代逐块互相关（IPCC）算法用于图像配准，重点在于优化处理速度并保持高空间精度。所测量的主要结果是每秒帧数（FPS）的处理速度以及图像配准的空间精度，通过配准图像与手动对齐图像之间的骰子系数进行量化。
当部署在边缘TPU上时，CNN模型与我们的图像配准算法相结合，以每秒14帧的速率处理视频流，这优于在其他标准硬件配置上实现的处理速率。IPCC算法有效地对齐了术前和术中图像，与手动配准相比显示出高精度。
本研究证明了在玻璃体视网膜手术中使用TPU加速的CNN来增强AR的可行性。
