Department of Surgery, Emory University School of Medicine, Atlanta, Georgia.
Department of Surgery, Emory University School of Medicine, Atlanta, Georgia.
J Surg Res. 2023 Nov;291:574-585. doi: 10.1016/j.jss.2023.07.008. Epub 2023 Aug 2.
Assessment of surgical resident technical performance is an integral component of any surgical training program. Timely assessment delivered in a structured format is a critical step to enhance technical skills, but residents often report that the quality and quantity of timely feedback received is lacking. Moreover, the absence of written feedback with specificity can allow residents to seemingly progress in their operative milestones as a junior resident, but struggle as they progress into their postgraduate year 3 and above. We therefore designed and implemented a web-based intraoperative assessment tool and corresponding summary "dashboard" to facilitate real-time assessment and documentation of technical performance.
A web form was designed leveraging a cloud computing platform and implementing a modified Ottawa Surgical Competency Operating Room Evaluation instrument; this included additional, procedure-specific criteria for select operations. A link to this was provided to residents via email and to all surgical faculty as a Quick Response code. Residents open and complete a portion of the form on a smartphone, then relinquish the device to an attending surgeon who then completes and submits the assessment. The data are then transferred to a secure web-based reporting interface; each resident (together with a faculty advisor) can then access and review all completed assessments.
The Assessment form was activated in June 2021 and formally introduced to all residents in July 2021, with residents required to complete at least one assessment per month. Residents with less predictable access to operative procedures (night float or Intensive Care Unit) were exempted from the requirement on those months. To date a total of 559 assessments have been completed for operations performed by 56 trainees, supervised by 122 surgical faculty and senior trainees. The mean number of procedures assessed per resident was 10.0 and the mean number per assessor was 4.6. Resident initiation of Intraoperative Assessments has increased since the tool was introduced and scores for technical and nontechnical performance reliably differentiate residents by seniority.
This novel system demonstrates that an online, resident-initiated technical assessment tool is feasible to implement and scale. This model's requirement that the attending enter performance ratings into the trainee's electronic device ensures that feedback is delivered directly to the trainee. Whether this aspect of our assessment ensures more direct and specific (and therefore potentially actionable) feedback is a focus for future study. Our use of commercial cloud computing services should permit cost-effective adoption of similar systems at other training programs.
评估外科住院医师的技术表现是任何外科培训计划的一个组成部分。及时以结构化的形式进行评估是提高技术技能的关键步骤，但住院医师经常报告说，他们收到的及时反馈的质量和数量都不足。此外，缺乏具体的书面反馈会让住院医师在初级住院医师阶段看似取得进展，但在进入研究生第 3 年及以上时会遇到困难。因此，我们设计并实施了一个基于网络的术中评估工具和相应的摘要“仪表板”，以方便实时评估和记录技术表现。
利用云计算平台设计了一个网络表单，并实施了经过修改的渥太华手术能力手术室评估工具；其中包括针对特定手术的其他程序特定标准。通过电子邮件向住院医师提供了该表单的链接，并将其作为快速响应码提供给所有外科教员。住院医师在智能手机上打开并完成表单的一部分，然后将设备交给主治外科医生，主治外科医生完成并提交评估。然后将数据传输到一个安全的基于网络的报告界面；每位住院医师（连同一名导师）都可以访问和查看所有已完成的评估。
评估表于 2021 年 6 月启用，并于 2021 年 7 月正式向所有住院医师推出，要求每位住院医师每月至少完成一次评估。在那些月份中，无法预测手术程序访问的住院医师（夜班或重症监护病房）可以免除该要求。迄今为止，共有 559 名住院医师为 56 名受训者进行的 559 次手术完成了评估，由 122 名外科教员和高级受训者进行了监督。每位住院医师评估的平均手术次数为 10.0 次，每位评估者的平均评估次数为 4.6 次。自该工具推出以来，住院医师开始进行术中评估的人数有所增加，并且技术和非技术表现的评分可靠地区分了住院医师的资历。
这个新系统表明，在线、住院医师发起的技术评估工具是可行的实施和扩展。这种模式要求主治医生将绩效评分输入学员的电子设备中，以确保反馈直接传达给学员。我们评估中的这一方面是否确保了更直接和具体（因此可能更具可操作性）的反馈，这是未来研究的重点。我们使用商业云计算服务应该允许其他培训计划以具有成本效益的方式采用类似的系统。
