D. Jurich is senior psychometrician, National Board of Medical Examiners, Philadelphia, Pennsylvania. M. Daniel is assistant dean for curriculum and associate professor, Departments of Emergency Medicine and Learning Health Sciences, University of Michigan Medical School, Ann Arbor, Michigan; ORCID: http://orcid.org/0000-0001-8961-7119. M. Paniagua is medical advisor, Test Development Services, National Board of Medical Examiners, and adjunct associate professor, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania. A. Fleming is associate dean for medical student affairs and associate professor of pediatrics, Vanderbilt University School of Medicine, Nashville, Tennessee. V. Harnik is associate dean for curriculum and associate professor, Department of Cell Biology, New York University School of Medicine, New York, New York. A. Pock is associate dean for curriculum and associate professor of medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland. A. Swan-Sein is director, Center for Education Research and Evaluation, and assistant professor of educational assessment, Columbia University Vagelos College of Physicians and Surgeons, New York, New York. M.A. Barone is vice president of licensure, National Board of Medical Examiners, Philadelphia, Pennsylvania. S.A. Santen is senior associate dean of evaluation, assessment and scholarship of learning, and professor of emergency medicine, Virginia Commonwealth University School of Medicine, Richmond, Virginia, and was previously assistant dean for educational research and quality improvement, University of Michigan Medical School, Ann Arbor, Michigan.
Acad Med. 2019 Mar;94(3):371-377. doi: 10.1097/ACM.0000000000002458.
Schools undergoing curricular reform are reconsidering the optimal timing of Step 1. This study provides a psychometric investigation of the impact on United States Medical Licensing Examination Step 1 scores of changing the timing of Step 1 from after completion of the basic science curricula to after core clerkships.
Data from four schools that recently moved the examination were analyzed in a pre-post format using examinee scores from three years before and after the change. The sample included scores from 2008 through 2016. Several confounders were addressed, including rising national scores and potential differences in cohort abilities using deviation scores and analysis of covariance (ANCOVA) controlling for Medical College Admission Test (MCAT) scores. A resampling procedure compared study schools' score changes versus similar schools' in the same time period.
The ANCOVA indicated postchange Step 1 scores were higher compared with prechange (adjusted difference = 2.67; 95% confidence interval: 1.50-3.83, P < .001; effect size = 0.14) after adjusting for MCAT scores and rising national averages. The average score increase in study schools was larger than changes seen in similar schools. Failure rates also decreased from 2.87% (n = 48) pre change to 0.39% (n = 6) post change (P < .001).
Results suggest moving Step 1 after core clerkships yielded a small increase in scores and a reduction in failure rates. Although these small increases are unlikely to represent meaningful knowledge gains, this demonstration of "noninferiority" may allow schools to implement significant curricular reforms.
正在进行课程改革的学校正在重新考虑 Step 1 的最佳时间。本研究通过心理测量学的方法调查了将 Step 1 的时间从基础科学课程完成后调整到核心实习后对美国医师执照考试 Step 1 成绩的影响。
使用三所学校在改革前后三年的考生成绩，以预-后格式对最近改变考试时间的四所学校的数据进行分析。该样本包括 2008 年至 2016 年的成绩。通过离差分数和协方差分析（ANCOVA）控制医学院入学考试（MCAT）分数，解决了几个混杂因素，包括全国分数的上升和潜在的不同队列能力差异。通过重采样程序，比较了研究学校与同一时期相同学校的分数变化。
在调整 MCAT 分数和全国平均分上升后，ANCOVA 表明与改革前相比，改革后的 Step 1 分数更高（调整后的差异=2.67；95%置信区间：1.50-3.83，P<0.001；效应量=0.14）。在研究学校中，平均分数的增加大于在相似学校中看到的变化。考试不及格率也从改革前的 2.87%（n=48）降至改革后的 0.39%（n=6）（P<0.001）。
结果表明，将 Step 1 推迟到核心实习后，分数略有提高，考试不及格率降低。尽管这些小的增加不太可能代表有意义的知识收益，但这种“非劣效性”的证明可能允许学校实施重大的课程改革。
