K.L. Arnhart is senior research analyst, Research and Data Integration, Federation of State Medical Boards, Euless, Texas; ORCID: http://orcid.org/0000-0001-9975-6358 .
M.M. Cuddy is measurement scientist, Center for Advanced Assessment, NBME, Philadelphia, Pennsylvania; ORCID: http://orcid.org/0000-0002-5756-9113 .
Acad Med. 2021 Sep 1;96(9):1319-1323. doi: 10.1097/ACM.0000000000004210.
The United States Medical Licensing Examination (USMLE) recently announced 2 policy changes: shifting from numeric score reporting on the Step 1 examination to pass/fail reporting and limiting examinees to 4 attempts for each Step component. In light of these policies, exam measures other than scores, such as the number of examination attempts, are of interest. Attempt limit policies are intended to ensure minimum standards of physician competency, yet little research has explored how Step attempts relate to physician practice outcomes. This study examined the relationship between USMLE attempts and the likelihood of receiving disciplinary actions from state medical boards.
The sample population was 219,018 graduates from U.S. and Canadian MD-granting medical schools who passed all USMLE Step examinations by 2011 and obtained a medical license in the United States, using data from the NBME and the Federation of State Medical Boards. Logistic regressions estimated how attempts on Steps 1, 2 Clinical Knowledge (CK), and 3 examinations influenced the likelihood of receiving disciplinary actions by 2018, while accounting for physician characteristics.
A total of 3,399 physicians (2%) received at least 1 disciplinary action. Additional attempts needed to pass Steps 1, 2 CK, and 3 were associated with an increased likelihood of receiving disciplinary actions (odds ratio [OR]: 1.07, 95% confidence interval [CI]: 1.01, 1.13; OR: 1.09, 95% CI: 1.03, 1.16; OR: 1.11, 95% CI: 1.04, 1.17, respectively), after accounting for other factors.
Physicians who took multiple attempts to pass Steps 1, 2 CK, and 3 were associated with higher estimated likelihood of receiving disciplinary actions. This study offers support for licensure and practice standards to account for physicians' USMLE attempts. The relatively small effect sizes, however, caution policy makers from placing sole emphasis on this relationship.
美国医师执照考试(USMLE)最近宣布了两项政策变化:从 Step 1 考试的数字分数报告改为通过/不通过报告,以及限制每位考生每个 Step 考试部分最多可参加 4 次考试。鉴于这些政策,除分数外,考试的其他措施,如考试次数,也很重要。尝试次数限制政策旨在确保医生的基本能力标准,但很少有研究探讨 Step 尝试与医生实践结果的关系。本研究考察了 USMLE 尝试次数与州医学委员会纪律处分的可能性之间的关系。
使用 NBME 和美国州医学委员会联合会的数据,对通过所有 USMLE Step 考试并于 2011 年获得美国行医执照的 219,018 名美国和加拿大医学博士授予学校的毕业生进行了样本分析。使用逻辑回归估计 Step 1、2 临床知识(CK)和 3 考试的尝试次数如何影响到 2018 年收到纪律处分的可能性,同时考虑到医生的特征。
共有 3,399 名医生(2%)至少收到 1 次纪律处分。通过 Step 1、2 CK 和 3 需要额外的尝试次数与收到纪律处分的可能性增加相关(优势比[OR]:1.07,95%置信区间[CI]:1.01,1.13;OR:1.09,95% CI:1.03,1.16;OR:1.11,95% CI:1.04,1.17,分别),在考虑了其他因素后。
多次尝试通过 Step 1、2 CK 和 3 的医生与更高的纪律处分可能性相关。本研究支持将执照和实践标准纳入医生的 USMLE 尝试次数。然而,相对较小的效应大小告诫政策制定者不要仅将重点放在这种关系上。