Abbott Kenneth L, Krumm Andrew E, Clark Michael J, Kendrick Daniel E, Kelley Jesse K, George Brian C
Department of Surgery, Center for Surgical Training and Research, University of Michigan, Ann Arbor, Michigan; University of Michigan Medical School, Ann Arbor, Michigan.
Department of Learning Health Sciences, University of Michigan, Ann Arbor, Michigan.
J Surg Educ. 2022 May-Jun;79(3):769-774. doi: 10.1016/j.jsurg.2021.12.010. Epub 2022 Jan 5.
Workplace-based assessment is increasingly prevalent in surgical education, especially for assessing operative skill. With current implementations, not all observed clinical performances are assessed, in part because trainees often have discretion about when they seek assessment. As a result, these samples of observed operative performances may not be representative of the full breadth of experience of surgical trainees. Therefore, analyses of these samples may be biased. We aimed to benchmark patterns of procedures logged in the SIMPL operative performance assessment system against records of trainee experience in Accreditation Council for Graduate Medical Education (ACGME) case logs.
We analyzed SIMPL longitudinal intraoperative performance assessments from categorical trainees in US general surgery residency programs. We compared overall patterns of how procedures are logged in SIMPL and in ACGME case logs using a Pearson correlation, and we examined differences in how individual procedures are logged in each system using Fisher's exact test.
Total procedure frequency from the SIMPL dataset was strongly correlated with total procedure frequency from ACGME case logs (r = 0.86, 95% CI 0.80-0.90). A subset of these procedures (10 of 116 procedures) was logged more frequently in the SIMPL dataset. These 10 procedures accounted for 56% of SIMPL observations and 30% of ACGME logged cases. Case complexity was comparable for assessments initiated by residents and faculty.
Samples of intraoperative performance ratings gathered using the SIMPL application largely resemble ACGME case logs. There is no evidence to indicate that residents preferentially select fewer complex cases for assessment.
基于工作场所的评估在外科教育中越来越普遍，尤其是用于评估手术技能。在当前的实施中，并非所有观察到的临床操作都得到评估，部分原因是学员在寻求评估的时间上往往有自主权。因此，这些观察到的手术操作样本可能无法代表外科住院医师的全部经验范围。所以，对这些样本的分析可能存在偏差。我们旨在将SIMPL手术操作评估系统中记录的手术模式与毕业后医学教育认证委员会（ACGME）病例日志中的住院医师经验记录进行对比。
我们分析了美国普通外科住院医师培训项目中分类住院医师的SIMPL纵向术中操作评估。我们使用Pearson相关性比较了SIMPL和ACGME病例日志中手术记录的总体模式，并使用Fisher精确检验检查了每个系统中单个手术记录的差异。
SIMPL数据集的总手术频率与ACGME病例日志的总手术频率高度相关（r = 0.86，95%可信区间0.80 - 0.90）。这些手术中的一个子集（116个手术中的10个）在SIMPL数据集中记录得更频繁。这10个手术占SIMPL观察的56%和ACGME记录病例的30%。住院医师和教员发起的评估的病例复杂性相当。
使用SIMPL应用程序收集的术中操作评分样本在很大程度上类似于ACGME病例日志。没有证据表明住院医师优先选择较简单的病例进行评估。
