Cassidy Douglas J, Chakraborty Saptarshi, Panda Nikhil, McKinley Sophia K, Mansur Arian, Hamdi Isra, Mullen John, Petrusa Emil, Phitayakorn Roy, Gee Denise
Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts.
Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, Manhattan, New York.
J Surg Educ. 2021 Jan-Feb;78(1):50-59. doi: 10.1016/j.jsurg.2020.06.038. Epub 2020 Jul 18.
Resident performance on the American Board of Surgery In-Training Examination (ABSITE) is used for evaluation of surgical knowledge and guides resident selection for institutional remediation programs. Remediation thresholds have historically been based on ABSITE percentile scores; however, this does not account for predictors that can impact a resident's exam performance. We sought to identify predictors of yearly ABSITE performance to help identify residents "at-risk" for performing below their expected growth trajectory.
The knowledge of the residents, as measured by standardized ABSITE scores, was modeled as a function of the corresponding postgraduate year via a linear mixed effects regression model. Additional model covariates included written USMLE-1-3 examination scores, gender, number of practice questions completed, and percentage correct of practice questions. For each resident, the predicted ABSITE standard score along with a 95% bootstrap prediction interval was obtained. Both resident-specific and population-level predictions for ABSITE standard scores were also estimated.
The study was conducted at a single, large academic medical center (Massachusetts General Hospital, Boston, MA).
Six years of general surgery resident score reports at a single institution between 2014 and 2019 were deidentified and analyzed.
A total of 376 score reports from 130 residents were analyzed. Covariates that had a significant effect on the model included USMLE-1 score (PGY1: p = 0.013; PGY2: p = 0.007; PGY3: p = 0.011), USMLE-2 score (PGY1: p < 0.001; PGY2: p < 0.001; PGY3: p < 0.001; PGY4: p < 0.001; PGY5: p = 0.032), male gender (PGY1: p = 0.003; PGY2: p < 0.001; PGY3: p < 0.001; PGY4: p = 0.008), and number of practice questions completed (p=0.003). Five residents were identified as having "fallen off" their predicted knowledge curve, including a single resident on 2 occasions. Population prediction curves were obtained at 7 different covariate percentile levels (5%, 10%, 25%, 50%, 75%, 90%, and 95%) that could be used to plot predicted resident knowledge progress.
Performance on USMLE-1 and -2 examinations, male gender, and number of practice questions completed were positive predictors of ABSITE performance. Creating residency-wide knowledge growth curves as well as individualized predictive ABSITE performance models allows for more efficient identification of residents potentially at risk for poor ABSITE performance and structured monitoring of surgical knowledge progression.
美国外科委员会住院医师培训考试（ABSITE）的成绩用于评估外科知识，并指导住院医师进入机构补救计划的选拔。补救阈值历来基于ABSITE百分位数得分；然而，这并未考虑到可能影响住院医师考试成绩的预测因素。我们试图确定每年ABSITE成绩的预测因素，以帮助识别那些“有风险”表现低于预期成长轨迹的住院医师。
通过线性混合效应回归模型，将以标准化ABSITE分数衡量的住院医师知识作为相应研究生年级的函数进行建模。其他模型协变量包括美国医师执照考试第1 - 3步的笔试成绩、性别、完成的练习题数量以及练习题的正确率。对于每位住院医师，获得了预测的ABSITE标准分数以及95%的自助法预测区间。还估计了住院医师个体和总体水平的ABSITE标准分数预测值。
该研究在一家大型学术医疗中心（马萨诸塞州波士顿的麻省总医院）进行。
对2014年至2019年期间一家机构六年的普通外科住院医师成绩报告进行了去识别化分析。
共分析了来自130名住院医师的376份成绩报告。对模型有显著影响的协变量包括美国医师执照考试第1步成绩（PGY1：p = 0.013；PGY2：p = 0.007；PGY3：p = 0.011）、美国医师执照考试第2步成绩（PGY1：p < 0.001；PGY2：p < 0.001；PGY3：p < 0.001；PGY4：p < 0.001；PGY5：p = 0.032）、男性性别（PGY1：p = 0.003；PGY2：p < 0.001；PGY3：p < 0.001；PGY4：p = 0.008）以及完成的练习题数量（p = 0.003）。确定有5名住院医师“偏离”了他们的预测知识曲线，其中一名住院医师出现了两次这种情况。在7个不同的协变量百分位数水平（5%、10%、25%、50%、75%、90%和95%）获得了总体预测曲线，可用于绘制预测的住院医师知识进展情况。
美国医师执照考试第1步和第2步的成绩、男性性别以及完成的练习题数量是ABSITE成绩的积极预测因素。创建全住院医师范围的知识增长曲线以及个性化的预测ABSITE成绩模型，能够更有效地识别可能在ABSITE考试中表现不佳的住院医师，并对手术知识进展进行结构化监测。
