Department of Surgery, Oregon Health & Science University, 3181 S.W. Sam Jackson Park Rd., Portland, OR, 97239, USA.
Minimally Invasive and Endoscopic Surgery at Montefiore Medical Center, New York, NY, USA.
Surg Endosc. 2024 Mar;38(3):1583-1591. doi: 10.1007/s00464-024-10677-2. Epub 2024 Feb 8.
Surgical videos coupled with structured assessments enable surgical training programs to provide independent competency evaluations and align with the American Board of Surgery's entrustable professional activities initiative. Existing assessment instruments for minimally invasive inguinal hernia repair (IHR) have limitations with regards to reliability, validity, and usability. A cross-sectional study of six surgeons using a novel objective, procedure-specific, 8-item competency assessment for minimally invasive inguinal hernia repair (IHR-OPSA) was performed to assess inter-rater reliability using a "safe" vs. "unsafe" scoring rubric.
The IHR-OPSA was developed by three expert IHR surgeons, field tested with five IHR surgeons, and revised based upon feedback. The final instrument included: (1) incision/port placement; (2) dissection of peritoneal flap (TAPP) or dissection of peritoneal flap (TEP); (3) exposure; (4) reducing the sac; (5) full dissection of the myopectineal orifice; (6) mesh insertion; (7) mesh fixation; and (8) operation flow. The IHR-OPSA was applied by six expert IHR surgeons to 20 IHR surgical videos selected to include a spectrum of hernia procedures (15 laparoscopic, 5 robotic), anatomy (14 indirect, 5 direct, 1 femoral), and Global Case Difficulty (easy, average, hard). Inter-rater reliability was assessed against Gwet's AC.
The IHR-OPSA inter-rater reliability was good to excellent, ranging from 0.65 to 0.97 across the eight items. Assessments of robotic procedures had higher reliability with near perfect agreement for 7 of 8 items. In general, assessments of easier cases had higher levels of agreement than harder cases.
A novel 8-item minimally invasive IHR assessment tool was developed and tested for inter-rater reliability using a "safe" vs. "unsafe" rating system with promising results. To promote instrument validity the IHR-OPSA was designed and evaluated within the context of intended use with iterative engagement with experts and testing of constructs against real-world operative videos.
手术视频与结构化评估相结合，使外科培训计划能够提供独立的能力评估，并与美国外科学院的可委托专业活动倡议保持一致。现有的微创腹股沟疝修补术（IHR）评估工具在可靠性、有效性和可用性方面存在局限性。本研究对六名外科医生进行了一项横断面研究，使用一种新的微创腹股沟疝修补术（IHR）客观、特定于程序的 8 项能力评估工具（IHR-OPSA），使用“安全”与“不安全”评分标准评估了组间可靠性。
IHR-OPSA 由三位微创腹股沟疝修补术专家开发，经过五位微创腹股沟疝修补术专家的现场测试，并根据反馈进行了修订。最终的工具包括：（1）切口/端口放置；（2）腹膜瓣切开（TAPP）或腹膜瓣切开（TEP）；（3）暴露；（4）减少疝囊；（5）完全切开耻骨肌孔；（6）网片插入；（7）网片固定；（8）手术流程。六位微创腹股沟疝修补术专家应用 IHR-OPSA 对 20 个微创腹股沟疝手术视频进行评估，这些视频包括各种疝手术（15 个腹腔镜手术，5 个机器人手术）、解剖结构（14 个间接疝，5 个直接疝，1 个股疝）和全球病例难度（简单、中等、困难）。使用 Gwet 的 AC 评估了组间可靠性。
IHR-OPSA 的组间可靠性良好至优秀，八项评估指标的范围为 0.65 至 0.97。八项评估指标中，有七项对机器人手术的评估具有较高的可靠性，几乎达到了完美的一致性。一般来说，较简单病例的评估一致性较高，而较困难病例的评估一致性较低。
开发了一种新的微创 IHR 评估工具，采用“安全”与“不安全”评分系统对其进行了组间可靠性测试，结果令人满意。为了提高仪器的有效性，IHR-OPSA 是在预期用途的背景下设计和评估的，并与专家进行了迭代式的合作，针对真实手术视频进行了结构测试。
