Bracken Keyna, Saleh Amr, Sandor Jeremy, Sibbald Matthew, Lee-Poy Micheal, Ngo Quang
Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Canada.
Med Sci Educ. 2024 Dec 4;35(1):351-358. doi: 10.1007/s40670-024-02197-4. eCollection 2025 Feb.
To better understand the link between formative assessments and progress difficulty, we conducted an analysis in the undergraduate MD program of the Michael G. DeGroote School of Medicine by comparing formative assessment scores on Short Answer Questions (SAQ) called Concept Application Exercises (CAE) with subsequent progress difficulty. CAE scores are designed to formatively assess knowledge translation. These scores are not formally incorporated into the progress decision at the end of each curricular unit, which is holistic in nature. Students are referred to a student progress remediation committee if they fail to meet the curricular objectives. We sought to investigate the following research question: Do short answer questions, in the form of CAEs, predict subsequent learner progress difficulty?
Data from the last four student cohorts of 2022-2025 were included. To address the predictive power of CAE score characteristics, a binary logistic regression model was constructed with remediation committee referral as the dependent variable and CAE score characteristics as the independent variable.
This study found that the average CAE score is the most powerful predictor of later progress difficulty, with each point drop in average score associated with a 37% increase in the odds of referral to the remediation committee.
These findings illustrate the predictive value of the SAQ to identify later progress difficulty.
为了更好地理解形成性评估与学习进展困难之间的联系，我们在迈克尔·G·德格罗特医学院的本科医学博士项目中进行了一项分析，比较了名为概念应用练习（CAE）的简答题的形成性评估分数与随后的学习进展困难情况。CAE分数旨在对知识转化进行形成性评估。这些分数在每个课程单元结束时的学习进展决策中并未正式纳入，该决策本质上是全面的。如果学生未达到课程目标，会被转介到学生学习进展补救委员会。我们试图研究以下研究问题：以CAE形式呈现的简答题能否预测随后学习者的学习进展困难？
纳入了2022 - 2025年最近四个学生队列的数据。为了探讨CAE分数特征的预测能力，构建了一个二元逻辑回归模型，将补救委员会转介作为因变量，将CAE分数特征作为自变量。
本研究发现，平均CAE分数是后期学习进展困难最有力的预测指标，平均分数每下降一分，被转介到补救委员会的几率就会增加37%。
这些发现说明了简答题在识别后期学习进展困难方面的预测价值。
