Use the right words: evaluating the effect of word choice and word count on quality of narrative feedback in ophthalmology competency-based medical education assessments.

BACKGROUND

The purpose of this study was to investigate the effect of word choice on the quality of narrative feedback in ophthalmology resident trainee assessments following the introduction of competency-based medical education at Queen's University.

METHODS

Assessment data from July 2017-December 2020 were retrieved from Elentra (Integrated Teaching and Learning Platform) and anonymized. Written feedback was assigned a Quality of Assessment for Learning (QuAL) score out of five based on this previously validated rubric. The correlation between QuAL score and specific coaching words was determined using a Spearman's Rho analysis. Independent samples t-tests were used to compare the QuAL score when a specific word was used, and when it was absent.

RESULTS

A total of 1997 individual assessments were used in this analysis. The number of times the identified coaching words were used within a comment was significantly and positively associated with the total QuAL score, with the exception of "next time" (rho=0.039, p=0.082), "read" (rho = 0.036, = 0.112), "read more" (rho = -0.025, = 0.256) and "review" (rho = -0.017, = 0.440). The strongest correlations were for "continue" (rho = 0.182, < 0.001), "try(ing)" (rho = 0.113, < 0.001) and "next step" (rho = 0.103, < 0.001). The mean value of the QuAL score increased when coaching words were used vs. not used with the largest mean difference of 1.44 ( < 0.001) for "reflect". A clear positive relationship was demonstrated between word count and QuAL score (rho = .556, < 0.001).

CONCLUSIONS

The use of certain coaching words in written comments may improve the quality of feedback.