Gong Zhiwei, Wan Bohua, Paranjape Jay N, Sikder Shameema, Patel Vishal M, Vedula S Swaroop
Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, MD, 21218, USA.
Department of Computer Science, Johns Hopkins University, Baltimore, MD, 21218, USA.
Int J Comput Assist Radiol Surg. 2025 May 22. doi: 10.1007/s11548-025-03406-0.
Assessment of intraoperative surgical skill is necessary to train surgeons and certify them for practice. The generalizability of deep learning models for video-based assessment (VBA) of surgical skill has not yet been evaluated. In this work, we evaluated one unsupervised domain adaptation (UDA) and three semi-supervised (SSDA) methods for generalizability of models for VBA of surgical skill in capsulorhexis by training on one dataset and testing on another.
We used two datasets, D99 and Cataract-101 (publicly available), and two state-of-the-art models for capsulorhexis. The models include a convolutional neural network (CNN) to extract features from video images, followed by a long short-term memory (LSTM) network or a transformer. We augmented the CNN and the LSTM with attention modules. We estimated accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC).
Maximum mean discrepancy (MMD) did not improve generalizability of CNN-LSTM but slightly improved CNN transformer. Among the SSDA methods, Group Distributionally Robust Supervised Learning improved generalizability in most cases.
Model performance improved with the domain adaptation methods we evaluated, but it fell short of within-dataset performance. Our results provide benchmarks on a public dataset for others to compare their methods.
评估术中手术技能对于培训外科医生并认证其执业资格至关重要。基于视频的手术技能评估（VBA）深度学习模型的通用性尚未得到评估。在这项工作中，我们通过在一个数据集上进行训练并在另一个数据集上进行测试，评估了一种无监督域适应（UDA）方法和三种半监督（SSDA）方法，以评估撕囊术中手术技能VBA模型的通用性。
我们使用了两个数据集，D99和白内障-101（公开可用），以及两种用于撕囊术的先进模型。这些模型包括一个卷积神经网络（CNN），用于从视频图像中提取特征，随后是一个长短期记忆（LSTM）网络或一个变换器。我们用注意力模块增强了CNN和LSTM。我们估计了准确率、敏感性、特异性和受试者操作特征曲线（AUC）下的面积。
最大均值差异（MMD）并没有提高CNN-LSTM的通用性，但略微提高了CNN变换器的通用性。在SSDA方法中，组分布鲁棒监督学习在大多数情况下提高了通用性。
我们评估的域适应方法提高了模型性能，但仍低于数据集内的性能。我们的结果在一个公共数据集上提供了基准，供其他人比较他们的方法。
