Brown Kristen C, Bhattacharyya Kiran D, Kulason Sue, Zia Aneeq, Jarc Anthony
Advanced Product Development, Intuitive Surgical, Inc., Norcross, Georgia, USA.
Visc Med. 2020 Dec;36(6):463-470. doi: 10.1159/000512437. Epub 2020 Oct 28.
A surgeon's technical skills are an important factor in delivering optimal patient care. Most existing methods to estimate technical skills remain subjective and resource intensive. Robotic-assisted surgery (RAS) provides a unique opportunity to develop objective metrics using key elements of intraoperative surgeon behavior which can be captured unobtrusively, such as instrument positions and button presses. Recent studies have shown that objective metrics based on these data (referred to as objective performance indicators [OPIs]) correlate to select clinical outcomes during robotic-assisted radical prostatectomy. However, the current OPIs remain difficult to interpret directly and, therefore, to use within structured feedback to improve surgical efficiencies.
We analyzed kinematic and event data from da Vinci surgical systems (Intuitive Surgical, Inc., Sunnyvale, CA, USA) to calculate values that can summarize the use of robotic instruments, referred to as OPIs. These indicators were mapped to broader technical skill categories of established training protocols. A data-driven approach was then applied to further sub-select OPIs that distinguish skill for each technical skill category within each training task. This subset of OPIs was used to build a set of logistic regression classifiers that predict the probability of expertise in that skill to identify targeted improvement and practice. The final, proposed feedback using OPIs was based on the coefficients of the logistic regression model to highlight specific actions that can be taken to improve.
We determine that for the majority of skills, only a small subset of OPIs (2-10) are required to achieve the highest model accuracies (80-95%) for estimating technical skills within clinical-like tasks on a porcine model. The majority of the skill models have similar accuracy as models predicting overall expertise for a task (80-98%). Skill models can divide a prediction into interpretable categories for simpler, targeted feedback.
We define and validate a methodology to create interpretable metrics for key technical skills during clinical-like tasks when performing RAS. Using this framework for evaluating technical skills, we believe that surgical trainees can better understand both what can be improved and how to improve.
外科医生的技术技能是提供最佳患者护理的重要因素。大多数现有的评估技术技能的方法仍然主观且资源密集。机器人辅助手术（RAS）提供了一个独特的机会，可以利用术中外科医生行为的关键要素来开发客观指标，这些要素可以在不引人注意的情况下被捕获，例如器械位置和按钮按压。最近的研究表明，基于这些数据的客观指标（称为客观性能指标[OPIs]）与机器人辅助根治性前列腺切除术中的特定临床结果相关。然而，当前的OPIs仍然难以直接解释，因此难以在结构化反馈中用于提高手术效率。
我们分析了来自达芬奇手术系统（直观外科公司，美国加利福尼亚州桑尼维尔）的运动学和事件数据，以计算可以总结机器人器械使用情况的值，即OPIs。这些指标被映射到既定培训方案中更广泛的技术技能类别。然后应用数据驱动的方法进一步从OPIs中进行子选择，以区分每个培训任务中每个技术技能类别的技能。这一子集的OPIs被用于构建一组逻辑回归分类器，该分类器预测该技能方面专业水平的概率，以识别有针对性的改进和实践。最终提出的使用OPIs的反馈基于逻辑回归模型的系数，以突出可以采取的具体改进行动。
我们确定，对于大多数技能，在猪模型上类似临床任务中估计技术技能时，仅需一小部分OPIs（2 - 10个）就能实现最高的模型准确率（80 - 95%）。大多数技能模型的准确率与预测任务整体专业水平的模型相似（80 - 98%）。技能模型可以将预测分为可解释的类别，以便进行更简单、有针对性的反馈。
我们定义并验证了一种方法，用于在进行RAS时为类似临床任务中的关键技术技能创建可解释的指标。使用这个框架来评估技术技能，我们相信外科实习生能够更好地理解哪些方面可以改进以及如何改进。
