1Department of Computer Science, USC Viterbi School of Engineering, Los Angeles, California.
2Department of Neurosurgery, Keck School of Medicine of USC, Los Angeles, California; and.
Neurosurg Focus. 2022 Apr;52(4):E11. doi: 10.3171/2022.1.FOCUS21652.
While the utilization of machine learning (ML) for data analysis typically requires significant technical expertise, novel platforms can deploy ML methods without requiring the user to have any coding experience (termed AutoML). The potential for these methods to be applied to neurosurgical video and surgical data science is unknown.
AutoML, a code-free ML (CFML) system, was used to identify surgical instruments contained within each frame of endoscopic, endonasal intraoperative video obtained from a previously validated internal carotid injury training exercise performed on a high-fidelity cadaver model. Instrument-detection performances using CFML were compared with two state-of-the-art ML models built using the Python coding language on the same intraoperative video data set.
The CFML system successfully ingested surgical video without the use of any code. A total of 31,443 images were used to develop this model; 27,223 images were uploaded for training, 2292 images for validation, and 1928 images for testing. The mean average precision on the test set across all instruments was 0.708. The CFML model outperformed two standard object detection networks, RetinaNet and YOLOv3, which had mean average precisions of 0.669 and 0.527, respectively, in analyzing the same data set. Significant advantages to the CFML system included ease of use, relatively low cost, displays of true/false positives and negatives in a user-friendly interface, and the ability to deploy models for further analysis with ease. Significant drawbacks of the CFML model included an inability to view the structure of the trained model, an inability to update the ML model once trained with new examples, and the inability for robust downstream analysis of model performance and error modes.
This first report describes the baseline performance of CFML in an object detection task using a publicly available surgical video data set as a test bed. Compared with standard, code-based object detection networks, CFML exceeded performance standards. This finding is encouraging for surgeon-scientists seeking to perform object detection tasks to answer clinical questions, perform quality improvement, and develop novel research ideas. The limited interpretability and customization of CFML models remain ongoing challenges. With the further development of code-free platforms, CFML will become increasingly important across biomedical research. Using CFML, surgeons without significant coding experience can perform exploratory ML analyses rapidly and efficiently.
虽然使用机器学习（ML）进行数据分析通常需要大量的技术专业知识，但新型平台可以部署 ML 方法，而无需用户具备任何编码经验（称为 AutoML）。这些方法应用于神经外科手术视频和手术数据科学的潜力尚不清楚。
使用无代码机器学习（CFML）系统对从高保真尸体模型上进行的先前验证的颈内动脉损伤训练练习中获得的内窥镜、经鼻手术过程中的视频的每一帧中包含的手术器械进行识别。使用 CFML 的器械检测性能与使用 Python 编程语言在同一手术视频数据集上构建的两个最先进的 ML 模型进行了比较。
CFML 系统无需使用任何代码即可成功摄取手术视频。总共使用了 31443 张图像来开发此模型；27223 张图像用于训练，2292 张图像用于验证，1928 张图像用于测试。所有器械在测试集上的平均精度均值为 0.708。CFML 模型在分析同一数据集时，优于两个标准的目标检测网络，RetinaNet 和 YOLOv3，其平均精度分别为 0.669 和 0.527。CFML 系统的显著优势包括易于使用、相对较低的成本、在用户友好的界面中显示真实/假阳性和阴性、以及轻松部署模型以进行进一步分析的能力。CFML 模型的显著缺点包括无法查看训练模型的结构、一旦使用新示例进行训练就无法更新 ML 模型、以及无法对模型性能和错误模式进行稳健的下游分析。
这是首次在对象检测任务中使用公共可用的手术视频数据集作为测试床描述 CFML 的基准性能的报告。与标准的基于代码的目标检测网络相比，CFML 超过了性能标准。这一发现对于寻求执行对象检测任务以回答临床问题、进行质量改进和开发新研究思路的外科医生科学家来说是令人鼓舞的。CFML 模型的有限可解释性和可定制性仍然是持续存在的挑战。随着无代码平台的进一步发展，CFML 将在生物医学研究中变得越来越重要。使用 CFML，没有大量编码经验的外科医生可以快速有效地执行探索性 ML 分析。
