Schumacher Daniel J, Kinnear Benjamin, Michelson Catherine, Stewart David A, Herman Bruce E, Wolfe Adam D, Winn Ariel, Boulais Jaclyn, Turner David A, Howell Heather B, Schwartz Alan
Acad Med. 2025 Jul 4. doi: 10.1097/ACM.0000000000006165.
Residency programs are increasingly interested in or required to assess residents using Accreditation Council for Graduate Medical Education (ACGME) milestones and specialty-defined entrustable professional activities (EPAs). The authors aimed to develop a model to predict individual residents' milestone levels based on their assigned EPA entrustment-supervision levels.
During 3 academic years from 2021 to 2024, the authors conducted a multisite prospective cohort study at 48 U.S. pediatric residency programs. Programs collected entrustment-supervision levels for the 17 general pediatrics EPAs and milestone levels for the 22 ACGME pediatric milestones for every resident biannually. EPA and milestone ratings were assigned by clinical competency committees. The first 4 of 6 biannual data reporting cycles were used to fit multilevel structural equation models and produce equations to generate, for each resident, predicted milestone levels based on EPA entrustment-supervision levels. They developed 2 models: one using 17 EPAs and one using 12 EPAs.
Data used for modeling represented 4,328 residents, with 164,886 total entrustment-supervision levels across the general pediatrics EPAs and 243,949 total milestone levels across the pediatric milestones. The fit of the round 1 to 4 model to the round 1 to 4 data (internal prediction) was excellent for both models, with comparative fit indexes of 0.982 (17 EPAs) and 0.981 (12 EPAs). The ability of the round 1 to 4 model to predict milestones for reporting cycles 5 and 6 (external prediction) was similar to the internal predictions, with correlation coefficients of 0.68 (17 EPAs) and 0.69 (12 EPAs) for round 5 and 0.72 (17 EPAs) and 0.68 (12 EPAs) for round 6.
This study demonstrates a strong ability to predict milestone levels based on EPA entrustment-supervision levels in a manner that enables meaningful use of EPAs and milestones in assessment efforts at residency programs.
住院医师培训项目越来越关注或被要求使用研究生医学教育认证委员会（ACGME）的里程碑和专业定义的可托付专业活动（EPA）来评估住院医师。作者旨在开发一种模型，根据分配给住院医师的EPA委托监督水平来预测其个人里程碑水平。
在2021年至2024年的3个学年中，作者在美国48个儿科住院医师培训项目中进行了一项多中心前瞻性队列研究。各项目每半年收集一次每位住院医师17项普通儿科EPA的委托监督水平以及22项ACGME儿科里程碑的里程碑水平。EPA和里程碑评级由临床能力委员会评定。6个半年期数据报告周期中的前4个用于拟合多层次结构方程模型，并生成方程，以便为每位住院医师根据EPA委托监督水平生成预测的里程碑水平。他们开发了2个模型：一个使用17项EPA，另一个使用12项EPA。
用于建模的数据代表4328名住院医师，普通儿科EPA的委托监督水平总计164886个，儿科里程碑的里程碑水平总计243949个。对于两个模型，第1至4轮模型与第1至4轮数据的拟合度（内部预测）都非常好，比较拟合指数分别为0.982（17项EPA）和0.981（12项EPA）。第1至4轮模型预测第5和第6个报告周期里程碑的能力（外部预测）与内部预测相似，第5轮的相关系数分别为0.68（17项EPA）和0.69（12项EPA），第6轮为0.72（17项EPA）和0.68（12项EPA）。
本研究表明，基于EPA委托监督水平预测里程碑水平的能力很强，能够在住院医师培训项目的评估工作中有效地利用EPA和里程碑。
