Department of Emergency Medicine, Medical College of Wisconsin, Milwaukee, Wisconsin.
Department of Emergency Medicine, John H. Stroger, Jr. Hospital of Cook County, Chicago, Illinois.
J Emerg Med. 2023 Mar;64(3):366-370. doi: 10.1016/j.jemermed.2023.01.003.
Interviews are an integral component of the residency selection process. Many programs use current residents as interviewers in addition to faculty. Although the reliability of interview scores between faculty members has been examined, little is known about the reliability between resident and faculty interviewers.
This study evaluates the reliability of residents as interviewers compared with faculty.
A retrospective analysis of interview scores from the 2020-2021 application cycle was performed at an emergency medicine (EM) residency program. Each applicant participated in five separate one-on-one interviews led by four faculty members and one senior resident. Interviewers assigned applicants a score from 0 to 10. Consistency between interviewers was measured using the intraclass correlation coefficient (ICC). Generalizability theory was used to measure variance components including applicant, interviewer, and rater type (resident vs. faculty) and their impact on scoring.
There were 250 applicants interviewed by 16 faculty members and 7 senior residents during the application cycle. The mean (SD) interview score given by resident interviewers was 7.10 (1.53) and the mean (SD) score given by faculty was 7.07 (1.69). There was no statistically significant difference between the pooled scores (p = 0.97). Reliability between interviewers was good to excellent (ICC = 0.90; 95% CI 0.88-0.92). The generalizability study showed most score variance was attributable to applicant characteristics and only 0.6% was attributable to interviewer or rater type (resident vs. faculty).
There was strong concordance between faculty and resident interview scores indicating reliability of EM resident scoring compared to faculty.
面试是住院医师选拔过程的一个组成部分。许多项目除了教职员工外，还会让现任住院医师担任面试官。虽然已经研究了教职员工之间的面试评分可靠性，但对于住院医师和教职员工面试官之间的可靠性知之甚少。
本研究评估住院医师作为面试官的可靠性与教职员工相比。
对一个急诊医学（EM）住院医师项目 2020-2021 年申请周期的面试评分进行回顾性分析。每位申请人参加了由四位教职员工和一位高级住院医师主持的五次单独的一对一面试。面试官从 0 到 10 分对申请人进行评分。使用组内相关系数（ICC）衡量面试官之间的一致性。使用广义理论测量方差分量，包括申请人、面试官和评分者类型（住院医师与教职员工）及其对评分的影响。
在申请周期内，有 250 名申请人接受了 16 名教职员工和 7 名高级住院医师的面试。住院医师面试官给出的平均（SD）面试分数为 7.10（1.53），教职员工给出的平均（SD）分数为 7.07（1.69）。两组分数之间没有统计学上的显著差异（p=0.97）。面试官之间的可靠性良好到优秀（ICC=0.90；95%CI 0.88-0.92）。广义理论研究表明，大部分评分差异归因于申请人的特征，只有 0.6%归因于面试官或评分者类型（住院医师与教职员工）。
教职员工和住院医师面试分数之间具有高度一致性，表明 EM 住院医师评分与教职员工相比具有可靠性。
