a College of Pharmacy , Qatar University , Doha , Qatar.
b Department of Educational Development and Research, Faculty of Health, Medicine, and Life Sciences , Maastricht University , Maastricht , Netherlands.
Teach Learn Med. 2019 Jun-Jul;31(3):298-306. doi: 10.1080/10401334.2018.1529570. Epub 2019 Feb 13.
: Authors examined the use of narrative comments for evaluation of student communications skills in a standardized, summative assessment (Objective Structured Clinical Examinations [OSCE]). : The use of narrative evaluations in workplace settings is gaining credibility as an assessment tool, but it is unknown how assessors convey judgments using narratives in high-stakes standardized assessments. The aim of this study was to explore constructs (i.e., performance dimensions), as well as linguistic strategies that assessors use to distinguish between poor and good students when writing narrative assessment comments of communication skills during an OSCE. : Eighteen assessors from Qatar University were recruited to write narrative assessment comments of communication skills for 14 students completing a summative OSCE. Assessors scored overall communication performance on a 5-point scale. Narrative evaluations for the top and bottom 2 performing students for each station (based on communication scores) were analyzed for linguistic strategies and constructs that informed assessment decisions. : Seventy-two narrative evaluations with 662 comments were analyzed. Most comments (77%) were written without the use of politeness strategies. A further 22% of comments were hedged. Hedging was used more commonly in poor performers, compared to good performers (30% vs. 15%, respectively). Overarching constructs of confidence, adaptability, patient safety, and professionalism were key dimensions that characterized the narrative evaluations of students' performance. : Results contribute to our understanding regarding the utility of narrative comments for summative assessment of communication skills. Assessors' comments could be characterized by the constructs of confidence, adaptability, patient safety, and professionalism when distinguishing between levels of student performance. Findings support the notion that judgments are arrived at by clustering sets of behaviors into overarching and meaningful constructs rather than by solely focusing on discrete behaviors. These results call for the development of better-anchored evaluation tools for communication assessment during OSCEs, constructively aligned with assessors' map of the reality of professional practice.
: 作者在标准化总结性评估（客观结构化临床考试 [OSCE]）中检查了叙事性评论在评估学生沟通技巧方面的使用。 : 在工作场所，叙事评估作为一种评估工具的可信度越来越高，但评估者在高风险标准化评估中使用叙事来传达判断的方式仍不得而知。本研究旨在探讨在 OSCE 中，评估者在撰写沟通技巧的叙事评估意见时，区分表现优秀和较差学生的结构（即表现维度）以及语言策略。 : 从卡塔尔大学招募了 18 名评估者，为完成总结性 OSCE 的 14 名学生撰写沟通技巧的叙事评估意见。评估者对整体沟通表现进行了 5 分制评分。根据沟通得分，对每个站点表现最好和最差的 2 名学生的叙事评估进行了分析，以了解评估决策的语言策略和结构。 : 分析了 72 份带有 662 条评论的叙事评估。大多数评论（77%）在撰写时没有使用礼貌策略。另外 22%的评论是模糊限制语。与表现良好的学生相比，表现较差的学生使用模糊限制语的情况更为常见（分别为 30%和 15%）。信心、适应性、患者安全和专业性等总体结构是评估学生表现的关键维度。 : 结果有助于我们了解叙事评论在总结性沟通技能评估中的效用。评估者的评论可以通过信心、适应性、患者安全和专业性等结构来区分学生的表现水平。研究结果支持这样一种观点，即判断是通过将一系列行为聚类为总体和有意义的结构来得出的，而不是仅仅关注离散的行为。这些结果呼吁为 OSCE 中的沟通评估开发更好的基于评估工具，与评估者对专业实践现实的理解相契合。
