Department of Neurology, Psychology and Psychiatry, Botucatu Medical School, UNESP-Universidade Estadual Paulista, Botucatu, São Paulo, Brazil.
Department of Surgery and Orthopedics, Botucatu Medical School, UNESP-Universidade Estadual Paulista, Botucatu, São Paulo, Brazil.
PLoS One. 2021 Sep 10;16(9):e0257293. doi: 10.1371/journal.pone.0257293. eCollection 2021.
Several methods have been proposed for analyzing differences between test scores, such as using mean scores, cumulative deviation, and mixed-effect models. Here, we explore the pooled analysis of retested Progress Test items to monitor the performance of first-year medical students who were exposed to a new curriculum design. This was a cross-sectional study of students in their first year of a medical program who participated in the annual interinstitutional Progress Tests from 2013 to 2019. We analyzed the performance of first-year students in the 2019 test and compared it with that of first-year students taking the test from 2013 to 2018 and encountering the same items. For each item, we calculated odds ratios with 95% confidence intervals; we also performed meta-analyses with fixed effects for each content area in the pooled analysis and presented the odds ratio (OR) with a 95% confidence interval (CI). In all, we used 63 items, which were divided into basic sciences, internal medicine, pediatrics, surgery, obstetrics and gynecology, and public health. Significant differences were found between groups in basic sciences (OR = 1.172 [CI95% 1.005 CI 1.366], p = 0.043) and public health (OR = 1.54 [CI95% CI 1.25-1.897], p < 0.001), which may reflect the characteristics of the new curriculum. Thus, pooled analysis of pretested items may provide indicators of different performance. This method may complement analysis of score differences on benchmark assessments.
已经提出了几种分析测试分数差异的方法，例如使用平均值、累积偏差和混合效应模型。在这里，我们探讨了重新测试进展测试项目的汇总分析，以监测接受新课程设计的一年级医学生的表现。这是一项对参加 2013 年至 2019 年年度机构间进展测试的医学项目一年级学生的横断面研究。我们分析了 2019 年测试中一年级学生的表现，并将其与参加 2013 年至 2018 年测试的一年级学生进行了比较，并遇到了相同的项目。对于每个项目，我们计算了具有 95%置信区间的优势比；我们还对汇总分析中的每个内容领域进行了固定效果的荟萃分析，并呈现了优势比（OR）和 95%置信区间（CI）。总共使用了 63 个项目，分为基础科学、内科、儿科学、外科学、妇产科和公共卫生。在基础科学（OR=1.172[CI95%1.005 CI 1.366]，p=0.043）和公共卫生（OR=1.54[CI95% CI 1.25-1.897]，p<0.001）方面，组间存在显著差异，这可能反映了新课程的特点。因此，预先测试项目的汇总分析可能提供不同表现的指标。这种方法可以补充基准评估中分数差异的分析。
