Atroshchenko Gennady V, Korup Lærke Riis, Hashemi Nasseh, Østergaard Lasse Riis, Tolsgaard Martin G, Rasmussen Sten
Department of Cardiothoracic Surgery, Aalborg University Hospital, Hobrovej 18-22, 9000, Aalborg, Denmark.
ROCnord Robotic Center Aalborg, Aalborg University Hospital, Aalborg, Denmark.
J Robot Surg. 2025 Jul 13;19(1):384. doi: 10.1007/s11701-025-02563-3.
To create a deep neural network capable of recognizing basic surgical actions and categorizing surgeons based on their skills using video data only. Nineteen surgeons with varying levels of robotic experience performed three wet lab tasks on a porcine model: robotic-assisted atrial closure, mitral stitches, and dissection of the thoracic artery. We used temporal labeling to mark two surgical actions: suturing and dissection. Each complete recording was annotated as either "novice" or "expert" based on the operator's experience. The network architecture combined a Convolutional Neural Network for extracting spatial features with a Long Short-Term Memory layer to incorporate temporal information. A total of 435 recordings were analyzed. The fivefold cross-validation yielded a mean accuracy of 98% for the action recognition (AR) and 79% for the skill assessment (SA) network. The AR model achieved an accuracy of 93%, with average recall, precision, and F1-score all at 93%. The SA network had an accuracy of 56% and a predictive certainty of 95%. Gradient-weighted Class Activation Mapping revealed that the algorithm focused on the needle, suture, and instrument tips during suturing, and on the tissue during dissection. AR network demonstrated high accuracy and predictive certainty, even with a limited dataset. The SA network requires more data to become a valuable tool for performance evaluation. When combined, these deep learning models can serve as a foundation for AI-based automated post-procedural assessments in robotic cardiac surgery simulation. ClinicalTrials.gov (NCT05043064).
创建一个深度神经网络，该网络仅使用视频数据就能识别基本的外科手术动作并根据外科医生的技能对其进行分类。19名具有不同机器人手术经验水平的外科医生在猪模型上执行了三项湿实验室任务：机器人辅助心房闭合、二尖瓣缝合和胸主动脉解剖。我们使用时间标记来标记两种手术动作：缝合和解剖。根据操作者的经验，每个完整记录被标注为“新手”或“专家”。该网络架构将用于提取空间特征的卷积神经网络与长短期记忆层相结合，以纳入时间信息。总共分析了435个记录。五折交叉验证得出动作识别（AR）网络的平均准确率为98%，技能评估（SA）网络的平均准确率为79%。AR模型的准确率达到93%，平均召回率、精确率和F1分数均为93%。SA网络的准确率为56%，预测确定性为95%。梯度加权类激活映射显示，该算法在缝合过程中关注针、缝线和器械尖端，在解剖过程中关注组织。即使数据集有限，AR网络也表现出高准确率和预测确定性。SA网络需要更多数据才能成为性能评估的有价值工具。这些深度学习模型结合使用时，可作为机器人心脏手术模拟中基于人工智能的术后自动评估的基础。ClinicalTrials.gov（NCT05043064）。
