Feldman Liane S, Hagarty Sarah E, Ghitulescu Gabriela, Stanbridge Donna, Fried Gerald M
Steinberg-Bernstein Centre for Minimally Invasive Surgery, McGill University, Montreal, Quebec, Canada.
J Am Coll Surg. 2004 Jan;198(1):105-10. doi: 10.1016/j.jamcollsurg.2003.08.020.
Technical skills of residents have traditionally been evaluated using subjective In-Training Evaluation Reports (ITERs). We have developed the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS), an objective measure of laparoscopic technical ability. The purpose of the study was to assess the concurrent validity of the MISTELS by exploring the relationship between MISTELS score and ITER assessment.
Fifty surgery residents were assessed on the MISTELS system. Concurrent ITER assessments of technical skill were collected, and the proportion of superior ratings for the year was calculated. Statistical comparisons were performed by ANOVA and chi-square analysis. The Pearson correlation coefficient was used to compare the scores in the MISTELS with the ITER ratings.
The 50 residents received 277 ITERs for the year, of which 103 (37%) were "superior," 170 (61%) "satisfactory," 4 (1%) "borderline," and 0 "unsatisfactory." The MISTELS score correlated moderately well with the proportion of superior ITER scores (r = 0.51, p < 0.01). Residents who passed the MISTELS had a higher proportion of superior ITER assessments than those who failed the MISTELS (p = 0.02), but residents who performed below their expected level on the MISTELS still received mainly satisfactory ITERs (82 +/- 18%).
The ITER assessment is poor at identifying residents with below-average technical skills. Residents who perform well in the MISTELS laparoscopic simulator also have better ITER evaluations, providing evidence for the concurrent validity of the MISTELS. Multiple assessment instruments are recommended for assessment of technical competency.
住院医师的技术技能传统上通过主观的培训期间评估报告（ITER）进行评估。我们开发了麦吉尔腹腔镜技能培训与评估无生命系统（MISTELS），这是一种评估腹腔镜技术能力的客观方法。本研究的目的是通过探讨MISTELS评分与ITER评估之间的关系来评估MISTELS的同时效度。
对50名外科住院医师进行MISTELS系统评估。收集技术技能的同期ITER评估，并计算当年优秀评级的比例。通过方差分析和卡方分析进行统计学比较。使用Pearson相关系数比较MISTELS中的分数与ITER评级。
这50名住院医师当年共收到277份ITER，其中103份（37%）为“优秀”，170份（61%）为“满意”，4份（1%）为“临界”，0份为“不满意”。MISTELS评分与优秀ITER评分的比例中度相关（r = 0.51，p < 0.01）。通过MISTELS的住院医师获得优秀ITER评估的比例高于未通过MISTELS的住院医师（p = 0.0），但在MISTELS中表现低于预期水平的住院医师仍主要获得满意的ITER（82±18%）。
ITER评估在识别技术技能低于平均水平的住院医师方面表现不佳。在MISTELS腹腔镜模拟器中表现出色的住院医师也有更好的ITER评估，为MISTELS的同时效度提供了证据。建议使用多种评估工具来评估技术能力。
