Breimer Gerben E, Haji Faizal A, Hoving Eelco W, Drake James M
Center for Image Guided Innovation and Therapeutic Intervention (CIGITI), The Hospital for Sick Children, Toronto, Ontario, Canada.
Childs Nerv Syst. 2015 Aug;31(8):1247-59. doi: 10.1007/s00381-015-2716-4. Epub 2015 May 1.
This study aims to develop and establish the content validity of multiple expert rating instruments to assess performance in endoscopic third ventriculostomy (ETV), collectively called the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT).
The important aspects of ETV were identified through a review of current literature, ETV videos, and discussion with neurosurgeons, fellows, and residents. Three assessment measures were subsequently developed: a procedure-specific checklist (CL), a CL of surgical errors, and a global rating scale (GRS). Neurosurgeons from various countries, all identified as experts in ETV, were then invited to participate in a modified Delphi survey to establish the content validity of these instruments. In each Delphi round, experts rated their agreement including each procedural step, error, and GRS item in the respective instruments on a 5-point Likert scale.
Seventeen experts agreed to participate in the study and completed all Delphi rounds. After item generation, a total of 27 procedural CL items, 26 error CL items, and 9 GRS items were posed to Delphi panelists for rating. An additional 17 procedural CL items, 12 error CL items, and 1 GRS item were added by panelists. After three rounds, strong consensus (>80% agreement) was achieved on 35 procedural CL items, 29 error CL items, and 10 GRS items. Moderate consensus (50-80% agreement) was achieved on an additional 7 procedural CL items and 1 error CL item. The final procedural and error checklist contained 42 and 30 items, respectively (divided into setup, exposure, navigation, ventriculostomy, and closure). The final GRS contained 10 items.
We have established the content validity of three ETV assessment measures by iterative consensus of an international expert panel. Each measure provides unique assessment information and thus can be used individually or in combination, depending on the characteristics of the learner and the purpose of the assessment. These instruments must now be evaluated in both the simulated and operative settings, to determine their construct validity and reliability. Ultimately, the measures contained in the NEVAT may prove suitable for formative assessment during ETV training and potentially as summative assessment measures during certification.
本研究旨在开发并确立多种专家评分工具的内容效度，以评估内镜下第三脑室造瘘术（ETV）的操作表现，这些工具统称为神经内镜脑室造瘘术评估工具（NEVAT）。
通过回顾当前文献、ETV视频以及与神经外科医生、进修医生和住院医生进行讨论，确定了ETV的重要方面。随后制定了三项评估措施：特定手术步骤检查表（CL）、手术错误检查表以及整体评分量表（GRS）。邀请了来自各个国家且均被认定为ETV专家的神经外科医生参与一项改良的德尔菲调查，以确立这些工具的内容效度。在每一轮德尔菲调查中，专家们根据5点李克特量表对他们对各自工具中每个手术步骤、错误以及GRS项目的认同程度进行评分。
17位专家同意参与该研究并完成了所有轮次的德尔菲调查。在生成项目后，总共向德尔菲小组成员提出了27项手术步骤CL项目、26项错误CL项目和9项GRS项目以供评分。小组成员又增加了17项手术步骤CL项目、12项错误CL项目和1项GRS项目。经过三轮调查，在35项手术步骤CL项目、29项错误CL项目和10项GRS项目上达成了强烈共识（>80%的认同率）。在另外7项手术步骤CL项目和1项错误CL项目上达成了中等共识（50 - 80%的认同率）。最终的手术步骤和错误检查表分别包含42项和30项（分为准备、暴露、导航、脑室造瘘和关闭）。最终的GRS包含10项。
我们通过国际专家小组的反复共识确立了三项ETV评估措施的内容效度。每项措施都提供了独特的评估信息，因此可以根据学习者的特点和评估目的单独使用或组合使用。现在必须在模拟和手术环境中对这些工具进行评估，以确定它们的结构效度和可靠性。最终，NEVAT中包含的措施可能被证明适用于ETV培训期间的形成性评估，并有可能作为认证期间的总结性评估措施。
