Liu Pin, Chen Shou-Yen, Chang Yu-Che, Ng Chip-Jin, Chaou Chung-Hsien
Department of Emergency Medicine, Chang Gung Memorial Hospital, Lin-Kou Medical Center, Taoyuan, Taiwan.
Department of Emergency Medicine, West Garden Hospital, Taipei, Taiwan.
Front Med (Lausanne). 2022 Mar 9;9:840721. doi: 10.3389/fmed.2022.840721. eCollection 2022.
In-training examination (ITE) has been widely adopted as an assessment tool to measure residents' competency. We incorporated different formats of assessments into the emergency medicine (EM) residency training program to form a multimodal, multistation ITE. This study was conducted to examine the cost and effectiveness of its different testing formats.
We conducted a longitudinal study in a tertiary teaching hospital in Taiwan. Nine EM residents were enrolled and followed for 4 years, and the biannual ITE scores were recorded and analyzed. Each ITE consisted of 8-10 stations and was categorized into four formats: multiple-choice question (MCQ), question and answer (QA), oral examination (OE), and high-fidelity simulation (HFS) formats. The learner satisfaction, validity, reliability, and costs were analyzed.
486 station scores were recorded during the 4 years. The numbers of MCQ, OE, QA, and HFS stations were 45 (9.26%), 90 (18.5%), 198 (40.7%), and 135 (27.8%), respectively. The overall Cronbach's alpha reached 0.968, indicating good overall internal consistency. The correlation with EM board examination was highest for HFS (ρ = 0.657). The average costs of an MCQ station, an OE station, and an HFS station were ~3, 14, and 21 times that of a QA station.
Multi-dimensional assessment contributes to good reliability. HFS correlates best with the final training exam score but is also the most expensive format among ITEs. Increased testing domains with various formats improve ITE's overall reliability. Program directors must understand each test format's strengths and limitations to bring forth the best combination of exams under the local context.
在职考试（ITE）已被广泛用作衡量住院医师能力的评估工具。我们将不同形式的评估纳入急诊医学（EM）住院医师培训计划，以形成多模式、多站式的ITE。本研究旨在探讨其不同测试形式的成本和效果。
我们在台湾一家三级教学医院进行了一项纵向研究。招募了9名EM住院医师，随访4年，记录并分析每半年一次的ITE成绩。每次ITE由8 - 10个站点组成，分为四种形式：多项选择题（MCQ）、问答（QA）、口试（OE）和高保真模拟（HFS）形式。分析了学习者满意度、效度、信度和成本。
4年期间记录了486个站点成绩。MCQ、OE、QA和HFS站点的数量分别为45个（9.26%）、90个（18.5%）、198个（40.7%）和135个（27.8%）。总体克朗巴哈系数达到0.968，表明总体内部一致性良好。HFS与EM委员会考试的相关性最高（ρ = 0.657）。一个MCQ站点、一个OE站点和一个HFS站点的平均成本分别约为一个QA站点的3倍、14倍和21倍。
多维度评估有助于提高信度。HFS与最终培训考试成绩的相关性最佳，但也是ITE中最昂贵的形式。增加多种形式测试领域可提高ITE的总体信度。项目主任必须了解每种测试形式的优缺点，以便在当地情况下实现最佳考试组合。
