Department of Neurological Surgery, Nippon Medical School Hospital.
Neurol Med Chir (Tokyo). 2021 Dec 15;61(12):750-757. doi: 10.2176/nmc.oa.2021-0191. Epub 2021 Oct 8.
The increase in minimally invasive surgery has led to a decrease in surgical experience. To date, there is only limited research examining whether skills are evaluated objectively and equally in simulation training, especially in microsurgery. The purpose of this study was to analyze the objectivity and equality of simulation evaluation results conducted in a contest format. A nationwide recruitment process was conducted to select study participants. Participants were recruited from a pool of qualified physicians with less than 10 years of experience. In this study, the simulation procedure consisted of incising a 1 mm thick blood vessel and suturing it with a 10-0 thread using a microscope. Initially, we planned to have the neurosurgical supervisors score the simulation procedure by direct observation. However, due to COVID-19, some study participants were unable to attend. Thus requiring some simulation procedures to be scored by video review. A total of 14 trainees participated in the study. The Cronbach's alpha coefficient among the scorers was 0.99, indicating a strong correlation. There was no statistically significant difference between the scores from the video review and direct observation judgments. There was a statistically significant difference (p <0.001) between the scores for some criteria. For the eight criteria, individual scorers assigned scores in a consistent pattern. However, this pattern differed between scorers indicating that some scorers were more lenient than others. The results indicate that both video review and direct observation methods are highly objective techniques evaluate simulation procedures.
微创手术的增加导致手术经验的减少。迄今为止，只有有限的研究检查了在模拟训练中技能是否得到客观和平等的评估，特别是在微创手术中。本研究的目的是分析竞赛形式下模拟评估结果的客观性和公平性。通过全国范围内的招募过程来选择研究参与者。参与者是从有不到 10 年经验的合格医生中招募的。在这项研究中，模拟程序包括用显微镜切开 1 毫米厚的血管并用 10-0 缝线缝合。最初，我们计划让神经外科主管通过直接观察来评分模拟程序。然而，由于 COVID-19，一些研究参与者无法参加。因此，需要一些模拟程序通过视频审查进行评分。共有 14 名学员参加了这项研究。评分者之间的 Cronbach's alpha 系数为 0.99，表明相关性很强。视频审查和直接观察判断的评分之间没有统计学上的显著差异。一些标准的评分存在统计学上的显著差异（p<0.001）。对于八个标准，个别评分者以一致的模式进行评分。然而，这种模式在评分者之间存在差异，表明一些评分者比其他评分者更宽容。结果表明，视频审查和直接观察方法都是高度客观的技术，可以评估模拟程序。
