University of California at San Francisco.
American Board of Internal Medicine, Philadelphia, Pennsylvania.
JAMA. 2016 Dec 6;316(21):2253-2262. doi: 10.1001/jama.2016.17357.
US internal medicine residency programs are now required to rate residents using milestones. Evidence of validity of milestone ratings is needed.
To compare ratings of internal medicine residents using the pre-2015 resident annual evaluation summary (RAES), a nondevelopmental rating scale, with developmental milestone ratings.
DESIGN, SETTING, AND PARTICIPANTS: Cross-sectional study of US internal medicine residency programs in the 2013-2014 academic year, including 21 284 internal medicine residents (7048 postgraduate-year 1 [PGY-1], 7233 PGY-2, and 7003 PGY-3).
Program director ratings on the RAES and milestone ratings.
Correlations of RAES and milestone ratings by training year; correlations of medical knowledge ratings with American Board of Internal Medicine (ABIM) certification examination scores; rating of unprofessional behavior using the 2 systems.
Corresponding RAES ratings and milestone ratings showed progressively higher correlations across training years, ranging among competencies from 0.31 (95% CI, 0.29 to 0.33) to 0.35 (95% CI, 0.33 to 0.37) for PGY-1 residents to 0.43 (95% CI, 0.41 to 0.45) to 0.52 (95% CI, 0.50 to 0.54) for PGY-3 residents (all P values <.05). Linear regression showed ratings differed more between PGY-1 and PGY-3 years using milestone ratings than the RAES (all P values <.001). Of the 6260 residents who attempted the certification examination, the 618 who failed had lower ratings using both systems for medical knowledge than did those who passed (RAES difference, -0.9; 95% CI, -1.0 to -0.8; P < .001; milestone medical knowledge 1 difference, -0.3; 95% CI, -0.3 to -0.3; P < .001; and medical knowledge 2 difference, -0.2; 95% CI, -0.3 to -0.2; P < .001). Of the 26 PGY-3 residents with milestone ratings indicating deficiencies on either of the 2 medical knowledge subcompetencies, 12 failed the certification examination. Correlation of RAES ratings for professionalism with residents' lowest professionalism milestone ratings was 0.44 (95% CI, 0.43 to 0.45; P < .001).
Among US internal medicine residents in the 2013-2014 academic year, milestone-based ratings correlated with RAES ratings but with a greater difference across training years. Both rating systems for medical knowledge correlated with ABIM certification examination scores. Milestone ratings may better detect problems with professionalism. These preliminary findings may inform establishment of the validity of milestone-based assessment.
现在要求美国内科住院医师培训计划使用里程碑来对住院医师进行评分。 需要有证据证明里程碑评分的有效性。
比较使用 2015 年前年度评估总结（RAES）的内科住院医师的评分，RAES 是一种非发展性评分量表，与发展性里程碑评分进行比较。
设计、地点和参与者： 2013-2014 学年美国内科住院医师培训计划的横断面研究，包括 21284 名内科住院医师（PGY-1 有 7048 名，PGY-2 有 7233 名，PGY-3 有 7003 名）。
方案主管在 RAES 和里程碑上的评分。
培训年限的 RAES 和里程碑评分的相关性；医学知识评分与美国内科委员会（ABIM）认证考试成绩的相关性；使用这两种系统评估不专业行为的评分。
相应的 RAES 评分和里程碑评分显示，随着培训年限的增加，相关性逐渐提高，从 0.31（95%CI，0.29 至 0.33）到 0.35（95%CI，0.33 至 0.37）的 PGY-1 住院医师到 0.43（95%CI，0.41 至 0.45）到 0.52（95%CI，0.50 至 0.54）的 PGY-3 住院医师（所有 P 值均<.05）。线性回归显示，使用里程碑评分的 PGY-1 和 PGY-3 年份之间的评分差异大于 RAES（所有 P 值均<.001）。在尝试参加认证考试的 6260 名居民中，有 618 名未通过考试，与通过考试的居民相比，他们的医学知识评分较低（RAES 差异，-0.9；95%CI，-1.0 至 -0.8；P<.001；里程碑医学知识 1 差异，-0.3；95%CI，-0.3 至 -0.3；P<.001；医学知识 2 差异，-0.2；95%CI，-0.3 至 -0.2；P<.001）。在 26 名 PGY-3 住院医师中，有 2 名在医学知识的任一分项能力上的里程碑评分显示存在不足，其中 12 名未能通过认证考试。RAES 评分的专业评分与居民最低专业里程碑评分的相关性为 0.44（95%CI，0.43 至 0.45；P<.001）。
在 2013-2014 学年的美国内科住院医师中，基于里程碑的评分与 RAES 评分相关，但随着培训年限的不同，评分差异更大。这两种医学知识评分系统均与 ABIM 认证考试成绩相关。里程碑评分可能更好地发现专业精神方面的问题。这些初步发现可能为基于里程碑的评估的有效性提供信息。
