M. Chakroun is a PhD student, Faculty of Medicine and Health Sciences, Université de Sherbrooke, Sherbrooke, Québec, Canada; ORCID: https://orcid.org/0000-0002-0518-1782 .
V.R. Dion was research assistant, Paul Grand'Maison de la Société des médecins de l'Université de Sherbrooke Research Chair in Medical Education, at the time of this work, and is now a first-year medical student, Université de Sherbrooke, Sherbrooke, Québec, Canada.
Acad Med. 2022 Nov 1;97(11):1699-1706. doi: 10.1097/ACM.0000000000004755. Epub 2022 May 24.
Narrative comments are increasingly used in assessment to document trainees' performance and to make important decisions about academic progress. However, little is known about how to document the quality of narrative comments, since traditional psychometric analysis cannot be applied. The authors aimed to generate a list of quality indicators for narrative comments, to identify recommendations for writing high-quality narrative comments, and to document factors that influence the quality of narrative comments used in assessments in higher education.
The authors conducted a scoping review according to Arksey & O'Malley's framework. The search strategy yielded 690 articles from 6 databases. Team members screened abstracts for inclusion and exclusion, then extracted numerical and qualitative data based on predetermined categories. Numerical data were used for descriptive analysis. The authors completed the thematic analysis of qualitative data with iterative discussions until they achieved consensus for the interpretation of the results.
After the full-text review of 213 selected articles, 47 were included. Through the thematic analysis, the authors identified 7 quality indicators, 12 recommendations for writing quality narratives, and 3 factors that influence the quality of narrative comments used in assessment. The 7 quality indicators are (1) describes performance with a focus on particular elements (attitudes, knowledge, skills); (2) provides a balanced message between positive elements and elements needing improvement; (3) provides recommendations to learners on how to improve their performance; (4) compares the observed performance with an expected standard of performance; (5) provides justification for the mark/score given; (6) uses language that is clear and easily understood; and (7) uses a nonjudgmental style.
Assessors can use these quality indicators and recommendations to write high-quality narrative comments, thus reinforcing the appropriate documentation of trainees' performance, facilitating solid decision making about trainees' progression, and enhancing the impact of narrative feedback for both learners and programs.
叙事性评论越来越多地用于评估，以记录学员的表现，并对其学业进展做出重要决策。然而，由于无法应用传统的心理计量分析，因此对于如何记录叙事性评论的质量知之甚少。作者旨在生成一份叙事性评论质量指标清单，确定撰写高质量叙事性评论的建议，并记录影响高等教育评估中使用的叙事性评论质量的因素。
作者根据 Arksey 和 O'Malley 的框架进行了范围综述。该搜索策略从 6 个数据库中产生了 690 篇文章。团队成员筛选摘要以确定纳入和排除标准，然后根据预定的类别提取数值和定性数据。数值数据用于描述性分析。作者通过迭代讨论完成了定性数据的主题分析，直到他们就结果的解释达成共识。
在对 213 篇选定文章进行全文审查后，有 47 篇被纳入。通过主题分析，作者确定了 7 个质量指标、12 条撰写高质量叙事评论的建议以及 3 个影响评估中使用的叙事性评论质量的因素。这 7 个质量指标是：（1）描述重点放在特定要素（态度、知识、技能）上的表现；（2）提供积极要素和需要改进的要素之间的平衡信息；（3）为学员提供如何提高表现的建议；（4）将观察到的表现与预期的表现标准进行比较；（5）为给出的分数/成绩提供依据；（6）使用清晰易懂的语言；以及（7）使用非评判性的风格。
评估者可以使用这些质量指标和建议来撰写高质量的叙事性评论，从而加强对学员表现的适当记录，为学员的进步做出坚实的决策，并增强叙事反馈对学员和项目的影响。
