Aryal Kamal Raj, Currow Chelise, Downey Sarah, Praseedom Raaj, Seager Alexander
Department of General Surgery, James Paget University Hospital, Great Yarmouth, United Kingdom.
Department of General Surgery, University of East Anglia, Norwich, United Kingdom.
Surg J (N Y). 2020 Mar 9;6(1):e49-e61. doi: 10.1055/s-0040-1708062. eCollection 2020 Jan.
In the United Kingdom, work-based assessments (WBAs) including procedure-based assessments (PBAs), case-based discussions (CBDs), clinical evaluation exercises (CEXs), and direct observation of procedural skills (DOPS) have been used in Higher General Surgical Training Program (HGSTP) since the introduction of Modernising Medical Careers. Although the Intercollegiate Surgical Curriculum Project states that they should be used for the formative development of trainees using feedback and reflection, there is no study to look at the perception of their usefulness and barriers in using them, particularly in HGSTP. The aim of this study is to investigate trainer's and trainee's perception of their usefulness, barriers in using them, and way forward for their improvement in HGSTP. This was a mixed method study. In phase I, after ethics committee approval, an online survey was sent to 83 trainers and 104 trainees, with a response rate of 33 and 37%, respectively, using Online Surveys (formerly Bristol Online Survey) from July 2018 to December 2018. After analysis of this result, in phase II, semistructured interviews were conducted with five trainees and five trainers who had volunteered to take part from phase I. Thematic analysis was performed to develop overarching themes. For professional formative development, 15% of the trainers and 53% of the trainees felt that WBAs had a low value. Among 4 WBAs-CEX, CBD, PBA, and DOPS-PBA was thought to be the most useful WBA by 52% trainers and 74% trainees. More trainers than trainees felt that it was being used as a formative tool (33 vs. 16%). The total number of WBAs thought to be required was between 20 and 40 per year, with 46% of the trainers and 53% of the trainees preferring these numbers. The thematic analysis generated four themes with subthemes in each: theme 1, "factors affecting usefulness," including the mode of validation, trainer/trainee engagement, and time spent in validating; theme 2, "doubt on utility" due to doubt on validity and being used as a tick-box exercise; theme 3, "pitfalls/difficulties" due to lack of time to validate, late validation, e-mail rather than face-to-face validation, trainer and trainee behavior, variability in feedback given, and emphasis on number than quality; and theme 4, "improvement strategies." The WBAs are not being used in a way they are supposed to be used. The perception of educational impact (Kirkpatrick levels 1 and 2) by trainers was more optimistic than by trainees. Improvements can be made by giving/finding more time, trainer training, more face-to-face validation, and better trainer trainee interactions.
在英国，自“医学职业现代化”计划实施以来，基于工作的评估（WBA），包括基于操作的评估（PBA）、案例讨论（CBD）、临床评估练习（CEX）和操作技能直接观察（DOPS），已被用于高等普通外科培训项目（HGSTP）。尽管跨学院外科课程项目指出，这些评估应用于通过反馈和反思促进学员的形成性发展，但尚无研究探讨对其有用性的看法以及使用过程中的障碍，尤其是在HGSTP中。本研究的目的是调查培训师和学员对其有用性的看法、使用中的障碍以及HGSTP中改进的方向。
这是一项混合方法研究。在第一阶段，经伦理委员会批准后，于2018年7月至2018年12月使用在线调查（原布里斯托尔在线调查）向83名培训师和104名学员发送了在线调查问卷，回复率分别为33%和37%。在分析该结果后，在第二阶段，对第一阶段自愿参与的五名学员和五名培训师进行了半结构化访谈。进行了主题分析以形成总体主题。
对于专业的形成性发展，15%的培训师和53%的学员认为WBA价值不大。在CEX、CBD、PBA和DOPS这四种WBA中，52%的培训师和74%的学员认为PBA是最有用的WBA。认为其被用作形成性工具的培训师比学员更多（33%对16%）。认为每年所需WBA总数在20至40次之间，46%的培训师和53%的学员倾向于这些次数。主题分析产生了四个主题，每个主题都有子主题：主题1，“影响有用性的因素”，包括验证模式、培训师/学员参与度以及验证所花费的时间；主题2，“对效用的怀疑”，原因是对有效性的怀疑以及被用作打勾练习；主题3，“陷阱/困难”，原因是缺乏验证时间、验证延迟、通过电子邮件而非面对面验证、培训师和学员的行为、反馈的变异性以及强调数量而非质量；主题4，“改进策略”。
WBA未按其应有的方式使用。培训师对教育影响（柯克帕特里克1级和2级）的看法比学员更乐观。可以通过提供/找到更多时间、培训师培训、更多面对面验证以及更好的培训师与学员互动来进行改进。
