Sydney Medical School - Northern, University of Sydney, Sydney, Australia.
Office of Medical Education, Sydney Medical School, University of Sydney, Sydney, Australia.
Med Educ. 2017 Apr;51(4):390-400. doi: 10.1111/medu.13151. Epub 2017 Jan 12.
Peer assessment of professional behaviour within problem-based learning (PBL) groups can support learning and provide opportunities to identify and remediate problem behaviours.
We investigated whether a peer assessment of learning behaviours in PBL is sufficiently valid to support decision making about student professional behaviours.
Data were available for two cohorts of students, in which each student was rated by all of their PBL group peers using a modified version of a previously validated scale. Following the provision of feedback to the students, their behaviours were again peer-assessed. A generalisability study was undertaken to calculate the students' professional behaviour scores, sources of error that impacted the reliability of the assessment, changes in student rating behaviour, and changes in mean scores after the delivery of feedback.
Peer assessment of professional learning behaviour was highly reliable for within-group comparisons (G = 0.81-0.87), but poor for across-group comparisons (G = 0.47-0.53). Feedback increased the range of ratings given by assessors and brought their mean ratings into closer alignment. More of the increased variance was attributable to assessee performance than to assessor stringency and hence there was a slight improvement in reliability, especially for comparisons across groups. Mean professional behaviour scores were unchanged.
Peer assessment of professional learning behaviours may be unreliable for decision making outside a PBL group. Faculty members should not draw conclusions from peer assessment about a student's behaviour compared with that of their peers in the cohort, and such a tool may not be appropriate for summative assessment. Health professional educators interested in assessing student professional behaviours in PBL groups might focus on opportunities for the provision of formative peer feedback and its impact on learning.
在基于问题的学习（PBL）小组中，同伴评估专业行为可以支持学习，并提供识别和纠正问题行为的机会。
我们调查了 PBL 中学生学习行为的同伴评估是否具有足够的有效性，以支持关于学生专业行为的决策。
我们有两个学生队列的数据，每个学生都被他们所有的 PBL 小组同伴使用以前验证过的量表的修改版进行评估。在向学生提供反馈后，他们的行为再次被同伴评估。我们进行了一项可推广性研究，以计算学生的专业行为分数、影响评估可靠性的误差来源、学生评分行为的变化以及反馈后平均分数的变化。
同伴对专业学习行为的评估在组内比较中具有高度可靠性（G = 0.81-0.87），但在组间比较中可靠性较差（G = 0.47-0.53）。反馈增加了评估者给出的评分范围，并使他们的平均评分更加一致。更多的增加方差归因于评估对象的表现，而不是评估者的严格程度，因此可靠性略有提高，特别是在组间比较中。专业行为的平均分数没有变化。
同伴对专业学习行为的评估可能不可靠，无法用于 PBL 小组之外的决策。教师不应根据同伴评估得出关于学生与同组学生行为的结论，并且该工具可能不适合总结性评估。对在 PBL 小组中评估学生专业行为感兴趣的健康专业教育者可能会关注提供形成性同伴反馈的机会及其对学习的影响。
