Sun Bo, Sun Zhen, Li Kexuan, Wang Xuehao, Wang Guotao, Song Wenfeng, Li Shuai, Hao Aimin, Xiao Yi
Research Unit of Virtual Body and Virtual Surgery Technologies, Division of Colorectal Surgery, Department of General Surgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 1 Shuai Fu Yuan, DongCheng District, Beijing, 100730, China; State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, No. 37 Xueyuan Road, Haidian District, Beijing, 100191, China.
Division of Colorectal Surgery, Department of General Surgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 1 Shuai Fu Yuan, Dongcheng District, Beijing, 100730, China.
Comput Methods Programs Biomed. 2024 Dec;257:108443. doi: 10.1016/j.cmpb.2024.108443. Epub 2024 Sep 28.
Accurate prostate dissection is crucial in transanal surgery for patients with low rectal cancer. Improper dissection can lead to adverse events such as urethral injury, severely affecting the patient's postoperative recovery. However, unclear boundaries, irregular shape of the prostate, and obstructive factors such as smoke present significant challenges for surgeons.
Our innovative contribution lies in the introduction of a novel video semantic segmentation framework, IG-Net, which incorporates prior surgical instrument features for real-time and precise prostate segmentation. Specifically, we designed an instrument-guided module that calculates the surgeon's region of attention based on instrument features, performs local segmentation, and integrates it with global segmentation to enhance performance. Additionally, we proposed a keyframe selection module that calculates the temporal correlations between consecutive frames based on instrument features. This module adaptively selects non-keyframe for feature fusion segmentation, reducing noise and optimizing speed.
To evaluate the performance of IG-Net, we constructed the most extensive dataset known to date, comprising 106 video clips and 6153 images. The experimental results reveal that this method achieves favorable performance, with 72.70% IoU, 82.02% Dice, and 35 FPS.
For the task of prostate segmentation based on surgical videos, our proposed IG-Net surpasses all previous methods across multiple metrics. IG-Net balances segmentation accuracy and speed, demonstrating strong robustness against adverse factors.
对于低位直肠癌患者，经肛门手术中准确的前列腺解剖至关重要。解剖不当可能导致尿道损伤等不良事件，严重影响患者术后恢复。然而，前列腺边界不清、形状不规则以及烟雾等阻碍因素给外科医生带来了重大挑战。
我们的创新贡献在于引入了一种新颖的视频语义分割框架IG-Net，该框架结合了先前的手术器械特征以进行实时精确的前列腺分割。具体而言，我们设计了一个器械引导模块，该模块基于器械特征计算外科医生的关注区域，进行局部分割，并将其与全局分割相结合以提高性能。此外，我们提出了一个关键帧选择模块，该模块基于器械特征计算连续帧之间的时间相关性。该模块自适应选择非关键帧进行特征融合分割，减少噪声并优化速度。
为了评估IG-Net的性能，我们构建了迄今为止最广泛的数据集，包括106个视频片段和6153张图像。实验结果表明，该方法取得了良好的性能，交并比为72.70%，Dice系数为82.02%，帧率为35帧每秒。
对于基于手术视频的前列腺分割任务，我们提出的IG-Net在多个指标上超越了以往所有方法。IG-Net在分割准确性和速度之间取得了平衡，对不利因素表现出强大的鲁棒性。
