E.S. Holmboe is chief research, milestones development and evaluation officer, Accreditation Council for Graduate Medical Education, Chicago, Illinois; ORCID: https://orcid.org/0000-0003-0108-6021. K. Yamazaki is senior analyst, Milestones, Accreditation Council for Graduate Medical Education, Chicago, Illinois. T.J. Nasca is president and chief executive officer, Accreditation Council for Graduate Medical Education, Chicago, Illinois, professor of medicine and molecular physiology, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania, and senior scholar, Department of Education, University of Illinois at Chicago School of Medicine, Chicago, Illinois; ORCID: https://orcid.org/0000-0003-0811-5462. S.J. Hamstra is vice president, Milestones Research and Evaluation, Accreditation Council for Graduate Medical Education, Chicago, Illinois, adjunct professor, Faculty of Education, University of Ottawa, Ottawa, Ontario, Canada, and adjunct professor, Department of Medical Education, Feinberg School of Medicine, Northwestern University, Chicago, Illinois; ORCID: https://orcid.org/0000-0002-0680-366X.
Acad Med. 2020 Jan;95(1):97-103. doi: 10.1097/ACM.0000000000002899.
To investigate the effectiveness of using national, longitudinal milestones data to provide formative assessments to identify residents at risk of not achieving recommended competency milestone goals by residency completion. The investigators hypothesized that specific, lower milestone ratings at earlier time points in residency would be predictive of not achieving recommended Level (L) 4 milestones by graduation.
In 2018, the investigators conducted a longitudinal cohort study of emergency medicine (EM), family medicine (FM), and internal medicine (IM) residents who completed their residency programs from 2015 to 2018. They calculated predictive values and odds ratios, adjusting for nesting within programs, for specific milestone rating thresholds at 6-month intervals for all subcompetencies within each specialty. They used final milestones ratings (May-June 2018) as the outcome variables, setting L4 as the ideal educational outcome.
The investigators included 1,386 (98.9%) EM residents, 3,276 (98.0%) FM residents, and 7,399 (98.0%) IM residents in their analysis. The percentage of residents not reaching L4 by graduation ranged from 11% to 31% in EM, 16% to 53% in FM, and 5% to 15% in IM. Using a milestone rating of L2.5 or lower at the end of post-graduate year 2, the predictive probability of not attaining the L4 milestone graduation goal ranged from 32% to 56% in EM, 32% to 67% in FM, and 15% to 36% in IM.
Longitudinal milestones ratings may provide educationally useful, predictive information to help individual residents address potential competency gaps, but the predictive power of the milestones ratings varies by specialty and subcompetency within these 3 adult care specialties.
调查使用国家纵向里程碑数据为形成性评估提供帮助的效果，以确定在 residency 完成之前处于无法达到推荐能力里程碑目标风险的 residents。研究人员假设 residency 早期特定的较低里程碑评级将预测未能在毕业前达到推荐的第 4 级里程碑。
在 2018 年，研究人员对完成 2015 年至 2018 年 residency 计划的急诊医学（EM）、家庭医学（FM）和内科（IM） residents 进行了一项纵向队列研究。他们为每个专业内的所有 subcompetencies 计算了特定里程碑评级阈值的预测值和优势比，以 6 个月的间隔进行调整，调整因素包括嵌套在程序内。他们使用最终的里程碑评分（2018 年 5 月至 6 月）作为因变量，将 L4 设定为理想的教育结果。
研究人员纳入了 1386 名（98.9%）EM residents、3276 名（98.0%）FM residents 和 7399 名（98.0%）IM residents。毕业时未达到 L4 的 residents 百分比在 EM 中从 11%到 31%不等，在 FM 中从 16%到 53%不等，在 IM 中从 5%到 15%不等。在 residency 毕业后第 2 年结束时使用里程碑评级为 L2.5 或更低，在 EM 中未达到 L4 里程碑毕业目标的预测概率从 32%到 56%不等，在 FM 中从 32%到 67%不等，在 IM 中从 15%到 36%不等。
纵向里程碑评级可能为个人 residents 提供有教育意义的预测信息，以帮助他们解决潜在的能力差距，但这些里程碑评级的预测能力因专业和这 3 个成人护理专业内的 subcompetency 而异。
