Wang Yaning, Wei Shuwen, Zuo Ruizhi, Kam Michael, Opfermann Justin D, Sunmola Idris, Hsieh Michael H, Krieger Axel, Kang Jin U
Department of Electrical and Computer Engineering, Johns Hopkins University, 3400 N. Charles St., Baltimore, MD 21218, USA.
Department of Mechanical Engineering, Johns Hopkins University, 3400 N. Charles St., Baltimore, MD 21218, USA.
Biomed Opt Express. 2024 Mar 25;15(4):2543-2560. doi: 10.1364/BOE.521652. eCollection 2024 Apr 1.
Anastomosis is a common and critical part of reconstructive procedures within gastrointestinal, urologic, and gynecologic surgery. The use of autonomous surgical robots such as the smart tissue autonomous robot (STAR) system demonstrates an improved efficiency and consistency of the laparoscopic small bowel anastomosis over the current da Vinci surgical system. However, the STAR workflow requires auxiliary manual monitoring during the suturing procedure to avoid missed or wrong stitches. To eliminate this monitoring task from the operators, we integrated an optical coherence tomography (OCT) fiber sensor with the suture tool and developed an automatic tissue classification algorithm for detecting missed or wrong stitches in real time. The classification results were updated and sent to the control loop of STAR robot in real time. The suture tool was guided to approach the object by a dual-camera system. If the tissue inside the tool jaw was inconsistent with the desired suture pattern, a warning message would be generated. The proposed hybrid multilayer perceptron dual-channel convolutional neural network (MLP-DC-CNN) classification platform can automatically classify eight different abdominal tissue types that require different suture strategies for anastomosis. In MLP, numerous handcrafted features (∼1955) were utilized including optical properties and morphological features of one-dimensional (1D) OCT A-line signals. In DC-CNN, intensity-based features and depth-resolved tissues' attenuation coefficients were fully exploited. A decision fusion technique was applied to leverage the information collected from both classifiers to further increase the accuracy. The algorithm was evaluated on 69,773 testing A-line data. The results showed that our model can classify the 1D OCT signals of small bowels in real time with an accuracy of 90.06%, a precision of 88.34%, and a sensitivity of 87.29%, respectively. The refresh rate of the displayed A-line signals was set as 300 Hz, the maximum sensing depth of the fiber was 3.6 mm, and the running time of the image processing algorithm was ∼1.56 s for 1,024 A-lines. The proposed fully automated tissue sensing model outperformed the single classifier of CNN, MLP, or SVM with optimized architectures, showing the complementarity of different feature sets and network architectures in classifying intestinal OCT A-line signals. It can potentially reduce the manual involvement of robotic laparoscopic surgery, which is a crucial step towards a fully autonomous STAR system.
吻合术是胃肠外科、泌尿外科和妇科手术中重建手术的常见且关键的部分。使用智能组织自主机器人（STAR）系统等自主手术机器人，相较于当前的达芬奇手术系统，腹腔镜小肠吻合术的效率和一致性得到了提高。然而，STAR工作流程在缝合过程中需要辅助人工监测，以避免漏缝或错缝。为了使操作员无需执行此监测任务，我们将光学相干断层扫描（OCT）光纤传感器与缝合工具集成，并开发了一种自动组织分类算法，用于实时检测漏缝或错缝。分类结果实时更新并发送到STAR机器人的控制回路。缝合工具由双摄像头系统引导以接近目标物体。如果工具钳口内的组织与所需的缝合模式不一致，将生成警告消息。所提出的混合多层感知器双通道卷积神经网络（MLP-DC-CNN）分类平台可以自动对八种不同的腹部组织类型进行分类，这些组织在吻合术中需要不同的缝合策略。在MLP中，利用了大量手工制作的特征（约1955个），包括一维（1D）OCT A线信号的光学特性和形态特征。在DC-CNN中，充分利用了基于强度的特征和深度分辨的组织衰减系数。应用决策融合技术利用从两个分类器收集的信息，以进一步提高准确性。该算法在69773条测试A线数据上进行了评估。结果表明，我们的模型可以实时对小肠的1D OCT信号进行分类，准确率为90.06%，精确率为88.34%，灵敏度为87.29%。显示的A线信号的刷新率设置为300 Hz，光纤的最大传感深度为3.6 mm，图像处理算法处理1024条A线的运行时间约为1.56 s。所提出的全自动组织传感模型优于具有优化架构的CNN、MLP或SVM单分类器，表明不同特征集和网络架构在对肠道OCT A线信号进行分类时具有互补性。它有可能减少机器人腹腔镜手术中的人工干预，这是迈向完全自主的STAR系统的关键一步。
