Thanitcul Chanon, Schnabel Sarah, Bartley George B, Lee Andrew G, Crump Hally, Fajardo Dale E, Peters Kathryn, Wilson Beth, Srikumaran Divya
Department of Ophthalmology and Visual Sciences, University of Michigan Kellogg Eye Center, Ann Arbor.
Department of Educational Psychology, University of Illinois Chicago, Chicago.
JAMA Ophthalmol. 2025 Jul 10. doi: 10.1001/jamaophthalmol.2025.1933.
The Ophthalmic Knowledge Assessment Program (OKAP) has been widely used to determine readiness to take the high-stakes Written Qualifying Examination (WQE). However, it is unclear how well OKAP performance can predict WQE outcomes.
To examine the association between the OKAP and WQE nationally.
DESIGN, SETTING, AND PARTICIPANTS: This was a retrospective cohort study conducted from 2017 to 2022. This national data analysis included residents from ophthalmology residency programs in the US. Study data were analyzed June 2023 to December 2024.
OKAP and WQE data.
The national OKAP examination data and the WQE data were matched. Candidates who had complete data for the OKAP during postgraduate year (PGY) 2, 3, and 4 and completed the WQE were included in the study. Two-sample t tests, logistic regression analysis, receiver operating characteristic curves, and confusion matrix were used to examine the association of various test scores (PGY-2 OKAP, PGY-3 OKAP, PGY-4 OKAP, minimum OKAP, maximum OKAP, average OKAP) as predictors for WQE outcomes (pass or fail). Multiple logistic regression analyses were also used to assess the association of gender, time since graduation, program size, and WQE pass/fail outcomes.
A total of 1597 residents (907 male [56.8%]) from 117 residency programs were included in this study. First-time pass rate of the WQE was 89% (1418 of 1597 residents). OKAP scores, particularly average OKAP, maximum OKAP, and PGY-4 OKAP scores (all had area under the curve = 0.88), were the most useful predictor of WQE outcomes. PGY-4 OKAP scores were used for the rest of the analysis for simplicity. A PGY-4 OKAP scaled score of 550 corresponded to 94% of candidates passing the WQE. Using PGY-4 OKAP scores to predict passing/failing, the WQE had 90.7% accuracy. Gender (odds ratio [OR], 1.50; 95% CI, 1.01-2.26; P = .048), program size (OR, 1.55; 95% CI, 0.98-2.41; P = .06), and time since residency graduation (OR, 1.00; 95% CI, 1.00-1.00; P = .14) were not included in the logistic regression model investigating the association of PGY-4 OKAP scores with the likelihood of passing the WQE.
Results suggest that OKAP examination scores were associated with passing the WQE after adjusting for program size, gender, and time since residency graduation. Further analyses may be warranted to predict failing the WQE so that potential interventions can be attempted.
眼科知识评估项目（OKAP）已被广泛用于确定参加高风险笔试资格考试（WQE）的准备情况。然而，尚不清楚OKAP成绩在多大程度上能够预测WQE的结果。
在全国范围内研究OKAP与WQE之间的关联。
设计、背景和参与者：这是一项于2017年至2022年进行的回顾性队列研究。这项全国性数据分析纳入了美国眼科住院医师培训项目的住院医师。研究数据于2023年6月至2024年12月进行分析。
OKAP和WQE数据。
将全国OKAP考试数据与WQE数据进行匹配。纳入在研究生第二年（PGY-2）、第三年（PGY-3）和第四年（PGY-4）期间拥有完整OKAP数据并完成WQE的考生。使用双样本t检验、逻辑回归分析、受试者工作特征曲线和混淆矩阵来研究各种考试成绩（PGY-2 OKAP、PGY-3 OKAP、PGY-4 OKAP、最低OKAP、最高OKAP、平均OKAP）作为WQE结果（通过或未通过）预测指标的关联。还使用多元逻辑回归分析来评估性别、毕业时间、项目规模与WQE通过/未通过结果之间的关联。
本研究纳入了来自117个住院医师培训项目的1597名住院医师（907名男性[56.8%]）。WQE的首次通过率为89%（1597名住院医师中的1418名）。OKAP成绩，尤其是平均OKAP、最高OKAP和PGY-4 OKAP成绩（曲线下面积均为0.88），是WQE结果最有用的预测指标。为简化起见，其余分析使用PGY-4 OKAP成绩。PGY-4 OKAP量表分数为550对应94%的考生通过WQE。使用PGY-4 OKAP成绩预测通过/未通过情况时，WQE的准确率为90.7%。在研究PGY-4 OKAP成绩与通过WQE可能性之间的关联的逻辑回归模型中，未纳入性别（优势比[OR]，1.50；95%置信区间，1.01 - 2.26；P = 0.048）、项目规模（OR，1.55；95%置信区间，0.98 - 2.41；P = 0.06）和住院医师毕业后时间（OR，1.00；95%置信区间，1.00 - 1.00；P = 0.14）。
结果表明，在调整项目规模、性别和住院医师毕业后时间后，OKAP考试成绩与通过WQE相关。可能需要进一步分析以预测WQE未通过情况，以便尝试采取潜在干预措施。
