Kocielnik Rafal, Yang Cherine H, Ma Runzhuo, Cen Steven Y, Wong Elyssa Y, Chu Timothy N, Knudsen J Everett, Wager Peter, Heard John, Ghaffar Umar, Anandkumar Anima, Hung Andrew J
Computing+Mathematical Sciences, California Institute of Technology, Pasadena, CA, USA.
Department of Urology, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
NPJ Digit Med. 2024 Dec 20;7(1):372. doi: 10.1038/s41746-024-01383-3.
Formative verbal feedback during live surgery is essential for adjusting trainee behavior and accelerating skill acquisition. Despite its importance, understanding optimal feedback is challenging due to the difficulty of capturing and categorizing feedback at scale. We propose a Human-AI Collaborative Refinement Process that uses unsupervised machine learning (Topic Modeling) with human refinement to discover feedback categories from surgical transcripts. Our discovered categories are rated highly for clinical clarity and are relevant to practice, including topics like "Handling and Positioning of (tissue)" and "(Tissue) Layer Depth Assessment and Correction [during tissue dissection]." These AI-generated topics significantly enhance predictions of trainee behavioral change, providing insights beyond traditional manual categorization. For example, feedback on "Handling Bleeding" is linked to improved behavioral change. This work demonstrates the potential of AI to analyze surgical feedback at scale, informing better training guidelines and paving the way for automated feedback and cueing systems in surgery.
术中实时的语言反馈对于调整实习生的行为和加速技能掌握至关重要。尽管其很重要，但由于难以大规模地捕捉和分类反馈，理解最佳反馈具有挑战性。我们提出了一种人机协作优化过程，该过程使用无监督机器学习（主题建模）并结合人工优化，从手术记录中发现反馈类别。我们发现的类别在临床清晰度方面得分很高且与实践相关，包括诸如“（组织）的处理和定位”以及“[组织解剖过程中的]（组织）层深度评估和校正”等主题。这些由人工智能生成的主题显著增强了对实习生行为变化的预测，提供了超越传统手动分类的见解。例如，关于“出血处理”的反馈与行为改善相关。这项工作展示了人工智能大规模分析手术反馈的潜力，为更好的培训指南提供信息，并为手术中的自动反馈和提示系统铺平道路。
