Department of Computer Science, Whiting School of Engineering, Johns Hopkins University, Baltimore, 21218, USA.
Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, 21218, USA.
Sci Rep. 2024 Nov 6;14(1):26912. doi: 10.1038/s41598-024-77176-1.
Accurate, unbiased, and reproducible assessment of skill is a vital resource for surgeons throughout their career. The objective in this research is to develop and validate algorithms for video-based assessment of intraoperative surgical skill. Algorithms to classify surgical video into expert or novice categories provide a summative assessment of skill, which is useful for evaluating surgeons at discrete time points in their training or certification of surgeons. Using a spatial-temporal neural network architecture, we tested the hypothesis that explicit supervision of spatial attention supervised by instrument tip locations improves the algorithm's generalizability to unseen dataset. The best performing model had an area under the receiver operating characteristic curve (AUC) of 0.88. Augmenting the network with supervision of spatial attention improved specificity of its predictions (with small changes in sensitivity and AUC) and led to improved measures of discrimination when tested with unseen dataset. Our findings show that explicit supervision of attention learned from images using instrument tip locations can improve performance of algorithms for objective video-based assessment of surgical skill.
准确、无偏且可重现的技能评估是外科医生整个职业生涯中的宝贵资源。本研究的目的是开发和验证基于视频的手术技能评估算法。将手术视频分类为专家或新手类别的算法提供了技能的总结性评估，这对于在培训或认证外科医生的离散时间点评估外科医生很有用。使用时空神经网络架构，我们检验了这样一个假设，即通过器械尖端位置进行的空间注意力的明确监督可以提高算法对未见数据集的泛化能力。表现最佳的模型的接收者操作特征曲线下面积（AUC）为 0.88。使用器械尖端位置从图像中学习的注意力的增强网络提高了其预测的特异性（敏感性和 AUC 略有变化），并在使用未见数据集进行测试时提高了区分度的度量。我们的研究结果表明，使用器械尖端位置从图像中学习的注意力的明确监督可以提高基于视频的手术技能客观评估算法的性能。
