Department of Cardiology and Pneumology, University Medical Centre Göttingen, Göttingen, Germany.
Division of Medical Education Research and Curriculum Development, University Medical Centre Göttingen, Göttingen, Germany.
Med Teach. 2021 May;43(5):524-530. doi: 10.1080/0142159X.2020.1867714. Epub 2021 Jan 27.
Low response rates threaten the reliability and validity of student evaluations of teaching. Previous research has shown that asking students to predict how satisfied their fellow students were with a course produces reliable results at lower response rates. The aim of this study was to investigate whether this prediction-based method can also be used to evaluate student learning outcome.
Before and after a cardiorespiratory module, 128 fourth-year medical students provided self-assessments and predictions of performance on 27 specific learning objectives and took formative tests on the respective contents. Pre-post performance gain was compared across all three modalities.
Formative exam results indicated a performance gain of 63.0%. Self-assessed and prediction-based performance gains were identical (67.8%) but both slightly overestimated actual performance gain. Irrespective of the method used, a response rate of 20% was sufficient to produce reliable results. Compared to male students, females greatly overestimated their peers' performance which led to inflated performance gain values.
Student self-assessments and predictions are equally valid sources of learning outcome measures, and low response rates are sufficient to produce stable results. When using a prediction-based approach, a tendency to overestimate learning outcome in female students needs to be taken into account.
低回应率会威胁教学评估的可靠性和有效性。先前的研究表明，让学生预测他们的同学对一门课程的满意度可以在较低的回应率下产生可靠的结果。本研究旨在探讨这种基于预测的方法是否也可用于评估学生的学习成果。
在心肺模块前后，128 名四年级医学生对 27 个具体学习目标进行了自我评估和预测，并对相应内容进行了形成性测试。比较了所有三种方式的前后表现增益。
形成性考试成绩表明表现增益为 63.0%。自我评估和基于预测的表现增益相同（67.8%），但都略微高估了实际表现增益。无论使用哪种方法，20%的回应率就足以产生可靠的结果。与男性学生相比，女性学生大大高估了他们同伴的表现，从而导致了 inflated performance gain values。
学生的自我评估和预测是学习成果测量的有效来源，较低的回应率足以产生稳定的结果。当使用基于预测的方法时，需要考虑到女性学生对学习成果高估的倾向。
