Centre for Image Guided Innovation and Therapeutic Intervention (CIGITI), The Hospital for Sick Children, Toronto, Ontario, Canada.
Department of Neuro-surgery, The Hospital for Sick Children, Toronto, Ontario, Canada.
Oper Neurosurg (Hagerstown). 2017 Feb 1;13(1):60-68. doi: 10.1227/NEU.0000000000001158.
Growing demand for transparent and standardized methods for evaluating surgical competence prompted the construction of the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT).
To provide validity evidence of the NEVAT by reporting on the tool's internal structure and its relationship with surgical expertise during simulation-based training.
The NEVAT was used to assess performance of trainees and faculty at an international neuroendoscopy workshop. All participants performed an endoscopic third ventriculostomy (ETV) on a synthetic simulator. Participants were simultaneously scored by 2 raters using the NEVAT procedural checklist and global rating scale (GRS). Evidence of internal structure was collected by calculating interrater reliability and internal consistency of raters' scores. Evidence of relationships with other variables was collected by comparing the ETV performance of experts, experienced trainees, and novices using Jonckheere's test (evidence of construct validity).
Thirteen experts, 11 experienced trainees, and 10 novices participated. The interrater reliability by the intraclass correlation coefficient for the checklist and GRS was 0.82 and 0.94, respectively. Internal consistency (Cronbach's α) for the checklist and the GRS was 0.74 and 0.97, respectively. Median scores with interquartile range on the checklist and GRS for novices, experienced trainees, and experts were 0.69 (0.58-0.86), 0.85 (0.63-0.89), and 0.85 (0.81-0.91) and 3.1 (2.5-3.8), 3.7 (2.2-4.3) and 4.6 (4.4-4.9), respectively. Jonckheere's test showed that the median checklist and GRS score increased with performer expertise ( P = .04 and .002, respectively).
This study provides validity evidence for the NEVAT to support its use as a standardized method of evaluating neuroendoscopic competence during simulation-based training.
对透明且标准化的手术能力评估方法的需求不断增长，促使我们构建了神经内镜脑室造瘘术评估工具（NEVAT）。
通过报告该工具在基于模拟的培训中的内部结构及其与手术专长的关系，为 NEVAT 的有效性提供证据。
在国际神经内镜研讨会上，使用 NEVAT 评估学员和教员的表现。所有参与者均在合成模拟器上进行内镜第三脑室造瘘术（ETV）。参与者同时由 2 名评分者使用 NEVAT 程序检查表和总体评分量表（GRS）进行评分。通过计算评分者之间的评分信度和内部一致性来收集内部结构的证据。通过比较专家、经验丰富的学员和新手的 ETV 表现，使用 Jonckheere 检验（结构有效性的证据）收集与其他变量关系的证据。
共有 13 名专家、11 名经验丰富的学员和 10 名新手参加了研究。检查表和 GRS 的组内相关系数的评分者间信度分别为 0.82 和 0.94。检查表和 GRS 的内部一致性（Cronbach's α）分别为 0.74 和 0.97。新手、经验丰富的学员和专家的检查表和 GRS 中位数（四分位距）分别为 0.69（0.58-0.86）、0.85（0.63-0.89）和 0.85（0.81-0.91）和 3.1（2.5-3.8）、3.7（2.2-4.3）和 4.6（4.4-4.9）。Jonckheere 检验表明，检查表和 GRS 评分的中位数随着执行者的专业水平的提高而增加（ P =.04 和.002）。
本研究为 NEVAT 的有效性提供了证据，支持其在基于模拟的培训中作为评估神经内镜能力的标准化方法的使用。
