Frey-Vogel Ariel S, Scott-Vernaglia Shannon E, Carter Lindsay P, Huang Grace C
From the Carl J. Shapiro Institute for Education and Research, Beth Israel Deaconess Medical Center (G.C.H.), and MassGeneral Hospital for Children (A.F.V., S.S.V., L.P.C.), Harvard Medical School, Boston, MA.
Simul Healthc. 2016 Aug;11(4):286-92. doi: 10.1097/SIH.0000000000000162.
Determining clinical competency on the milestones requires direct observation of residents, which is difficult for faculty members who are also providing patient care. Simulation can potentially represent an effective standardized tool for high-stakes assessment. Using a longitudinal simulation curriculum with formative and summative components, we conducted a pilot investigation to examine whether (1) performance on the formative cases predicted performance on the summative cases and (2) performance on the summative cases correlated with the clinical competency committee's (CCC) milestone placement.
We developed 6 formative and 6 matched summative cases for pediatric interns that covered core pediatric topics. The interns progressed through the formative cases in pairs during the course of the academic year and then through the summative cases back to back individually at the end of the year. The interns were evaluated using a competency-based simulation evaluation. We determined the relationship between the formative and summative scores and between the summative scores and those from the CCC.
The relationship between formative and summative scores was not statistically significant. There was a statistically significant relationship between summative and CCC scores.
Significant methodological limitations preclude definite conclusions about the predictive power of simulation cases for Pediatric Milestones-based assessment. However, our work is an example of how simulation is a potentially useful tool for assessing residents' skill development on the Pediatric Milestones. More rigorous research is needed to determine the extent to which simulation can be used for high-stakes, milestones-based assessment.
依据这些里程碑来确定临床能力需要对住院医师进行直接观察，这对于同时还提供患者护理的教员来说很困难。模拟可能是一种用于高风险评估的有效标准化工具。我们采用了一个包含形成性和总结性部分的纵向模拟课程，进行了一项试点调查，以检验：（1）形成性病例的表现是否能预测总结性病例的表现；（2）总结性病例的表现与临床能力委员会（CCC）的里程碑评定是否相关。
我们为儿科实习生开发了6个形成性病例和6个匹配的总结性病例，涵盖了儿科核心主题。实习生在学年期间成对完成形成性病例，然后在年底单独连续完成总结性病例。使用基于能力的模拟评估对实习生进行评价。我们确定了形成性分数与总结性分数之间以及总结性分数与CCC分数之间的关系。
形成性分数与总结性分数之间的关系无统计学意义。总结性分数与CCC分数之间存在统计学意义的关系。
显著的方法学局限性使得无法就模拟病例对基于儿科里程碑的评估的预测能力得出明确结论。然而，我们的工作展示了模拟如何可能成为评估住院医师在儿科里程碑方面技能发展的有用工具。需要更严格的研究来确定模拟可用于高风险、基于里程碑的评估的程度。
