Department of Otorhinolaryngology, Head & Neck Surgery and Audiology, Rigshospitalet, 9 Blegdamsvej, 2100, Copenhagen Ø, Denmark.
The Simulation Centre, Copenhagen Academy for Medical Education and Simulation (CAMES), The Capital Region of Denmark, Copenhagen, Denmark.
Eur Arch Otorhinolaryngol. 2022 Jan;279(1):127-136. doi: 10.1007/s00405-021-06632-9. Epub 2021 Feb 19.
To develop and gather validity evidence for a novel tool for assessment of cochlear implant (CI) surgery, including virtual reality CI surgery training.
Prospective study gathering validity evidence according to Messick's framework. Four experts developed the CI Surgery Assessment Tool (CISAT). A total of 35 true novices (medical students), trained novices (residents) and CI surgeons performed two CI-procedures each in the Visible Ear Simulator, which were rated by three blinded experts. Classical test theory and generalizability theory were used for reliability analysis.
The CISAT significantly discriminated between the three groups (p < 0.001). The generalizability coefficient was 0.76 and most of the score variance (53.3%) was attributable to the participant and only 6.8% to the raters. When exploring a standard setting for CI surgery, the contrasting groups method suggested a pass/fail score of 36.0 points (out of 55), but since the trained novices performed above this, we propose using the mean CI surgeon performance score (45.3 points).
Validity evidence for simulation-based assessment of CI performance supports the CISAT. Together with the standard setting, the CISAT might be used to monitor progress in competency-based training of CI surgery and to determine when the trainee can advance to further training.
开发并收集一种新型耳蜗植入（CI）手术评估工具的有效性证据，包括虚拟现实 CI 手术培训。
根据 Messick 框架进行的前瞻性研究，以收集有效性证据。四位专家开发了 CI 手术评估工具（CISAT）。共有 35 名真正的新手（医学生）、受训新手（住院医师）和 CI 外科医生在可视耳模拟器中各完成了两次 CI 手术，由三位盲评专家进行评分。使用经典测试理论和可概括性理论进行可靠性分析。
CISAT 显著区分了三组（p<0.001）。可概括系数为 0.76，评分方差的大部分（53.3%）归因于参与者，只有 6.8%归因于评分者。在探索 CI 手术的标准设定时，对比组方法建议 36.0 分（满分 55 分）及格/不及格，但由于受训新手的得分高于此，因此我们建议使用 CI 外科医生的平均表现得分（45.3 分）。
基于模拟的 CI 性能评估的有效性证据支持 CISAT。与标准设定相结合，CISAT 可用于监测基于能力的 CI 手术培训的进展，并确定学员何时可以进阶到进一步的培训。
