A. Schwartz is the Michael Reese Endowed Professor of Medical Education and research professor, pediatrics, University of Illinois College of Medicine, and network director, Association of Pediatric Program Directors (APPD) Longitudinal Educational Assessment Research Network (LEARN), Chicago, Illinois.
D.F. Balmer is associate professor, pediatrics, The Children's Hospital of Pennsylvania and University of Pennsylvania, Philadelphia, Pennsylvania.
Acad Med. 2020 Nov;95(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 59th Annual Research in Medical Education Presentations):S95-S102. doi: 10.1097/ACM.0000000000003638.
To evaluate response process validity evidence for clinical competency committee (CCC) assessments of first-year residents on a subset of General Pediatrics Entrustable Professional Activities (EPAs) and milestones in the context of a national pilot of competency-based, time-variable (CBTV) advancement from undergraduate to graduate medical education.
Assessments of 2 EPAs and 8 milestones made by the trainees' actual CCCs and 2 different blinded "virtual" CCCs for 48 first-year pediatrics residents at 4 residency programs between 2016 and 2018 were compared. Residents had 3 different training paths from medical school to residency: time-variable graduation at the same institution as their residency, time-fixed graduation at the same institution, or time-fixed graduation from a different institution. Assessments were compared using ordinal mixed-effects models.
Actual CCCs assigned residents higher scores than virtual CCCs on milestones and one EPA's supervision levels. Residents who graduated from a different institution than their residency received lower milestone ratings than either group from the same institution; CBTV residents received higher ratings on one milestone (ICS4) and similar ratings on all others compared with non-CBTV residents who completed medical school at the same institution.
First-year residents who graduated from CBTV medical school programs were assessed as having the same level of competence as residents who graduated from traditional medical school programs, but response process evidence suggests that members of CCCs may also draw on undocumented personal knowledge of the learner to draw conclusions about resident competence.
在全国范围内进行基于能力的、时间可变（CBTV）从本科到研究生医学教育的试点项目中，评估临床能力委员会（CCC）对第一年住院医师在一般儿科委托专业活动（EPA）和里程碑方面的评估过程有效性证据。
将 2016 年至 2018 年间在 4 个住院医师培训项目中的 48 名第一年儿科住院医师的实际 CCC 及其 2 个不同的盲目的“虚拟”CCC 对 2 个 EPA 和 8 个里程碑进行评估。住院医师有 3 种不同的从医学院到住院医师的培训途径：与住院医师相同机构的时间可变毕业、相同机构的时间固定毕业或不同机构的时间固定毕业。使用有序混合效应模型对评估进行比较。
实际 CCC 对里程碑和一个 EPA 的监督水平给予住院医师的评分高于虚拟 CCC。与同一机构的其他两组相比，从不同机构毕业的住院医师获得的里程碑评分较低；与在同一机构完成医学院学业的非 CBTV 住院医师相比，CBTV 住院医师在一个里程碑（ICS4）上的评分较高，在所有其他里程碑上的评分相似。
从 CBTV 医学院毕业的第一年住院医师的评估水平与从传统医学院毕业的住院医师相同，但反应过程证据表明，CCC 成员也可能利用未记录的个人学习者知识来得出关于住院医师能力的结论。
