Surgical Service, Urology Section, Veterans Affairs Boston Healthcare System, Jamaica Plain, Massachusetts 02130, USA.
J Urol. 2011 Aug;186(2):634-7. doi: 10.1016/j.juro.2011.03.126. Epub 2011 Jun 16.
The American Urological Association In-Service Examination and the American Board of Urology Qualifying Examination are written multiple choice tests that cover all domains in urology. We investigated whether In-Service Examination performance could identify chief residents who scored in the lowest quartile on the Qualifying Examination.
All urology chief residents in the United States and Canada in 2008 and 2009 were eligible to participate in this study. In-Service Examination 2008 and Qualifying Examination 2009 performance data were obtained from the American Urological Association and American Board of Urology, respectively. Data were analyzed with the Pearson correlation and descriptive statistics.
Of the 257 American and Canadian chief residents who completed the Qualifying Examination 2009, 194 (75%) enrolled in this study and were included in analysis. Overall In-Service Examination 2008 scores correlated significantly with Qualifying Examination 2009 scores (r=0.55, p<0.001), accounting for 30% of score variance. Substantial variability in In-Service Examination-Qualifying Examination rankings was notable among individual residents. An In-Service Examination 2008 cutoff percentile rank of 40% identified chief residents in the lowest quartile on the Qualifying Examination 2009 with 71% sensitivity, 77% specificity, and a likelihood ratio of 3.1 and 0.4 (positive and negative likelihood ratios, respectively).
The substantial variability of In-Service Examination-Qualifying Examination performance among individual chief residents limits In-Service Examination predictive utility. A single In-Service Examination score should not be used to make a high stakes judgment about an individual resident. In-Service Examination scores should be used as 1 part of an overall evaluation program to prospectively identify residents who could benefit from additional educational support.
美国泌尿外科学会在职考试和美国泌尿外科学委员会资格考试都是涵盖泌尿外科学所有领域的多项选择题考试。我们研究了在职考试的表现是否可以识别出在资格考试中得分最低的住院总医师。
2008 年和 2009 年，美国和加拿大的所有泌尿科住院总医师都有资格参加这项研究。从美国泌尿外科学会和美国泌尿外科学委员会分别获得了 2008 年在职考试和 2009 年资格考试的成绩数据。采用 Pearson 相关分析和描述性统计分析数据。
在参加 2009 年资格考试的 257 名美国和加拿大住院总医师中，有 194 名（75%）参加了这项研究并纳入分析。总体而言，2008 年在职考试成绩与 2009 年资格考试成绩显著相关（r=0.55，p<0.001），占分数方差的 30%。个别住院医师的在职考试-资格考试排名存在很大差异。2008 年在职考试的 40%百分位排名可识别出在 2009 年资格考试中得分最低的四分之一住院总医师，其灵敏度为 71%，特异性为 77%，优势比为 3.1 和 0.4（阳性和阴性似然比，分别）。
个别住院医师的在职考试-资格考试成绩存在很大差异，限制了在职考试的预测能力。单个在职考试分数不应用于对个别住院医师做出高风险判断。在职考试成绩应作为整体评估计划的一部分，前瞻性地识别出需要额外教育支持的住院医师。
