MaReS (Magdeburg Reflective Writing Scoring Rubric for Feedback) - development of a feedback method for reflective writing in health professions education: A pilot study in veterinary medicine.

AIM

The aim of the study was to develop a scoring rubric that provides valuable feedback to students and to gather evidence for its construct validity.

METHODOLOGY

The Magdeburg Reflective Writing Feedback and Scoring Rubric (MaReS) was developed in an iterative process following a symposium on reflection by a committee of the "DACH Association for Medical Education (GMA)" in June 2016. 25 essays written by 13 veterinary students were assessed by three independent raters with MaReS and by two raters with the REFLECT rubric in two runs (13 and twelve essays). Validity evidence was gathered referring to the following of Messick's components of construct validity: content (rubric development), response process (rater manual, rater training, rating time, students' evaluation), internal structure (inter-rater reliability, IRR), and relationship to other variables (comparison of the rating with the REFLECT rubric and a global rating scale).

RESULTS

The analytic rubric comprises twelve items that are rated on three-point rating scales. The authors developed an assignment with guiding questions for students and a rater manual. Results for free marginal kappa of the items of MaReS ranged from -0.08 to 0.77 for the first set of reflective essays and from 0.13 to 0.75 for the second set. Correlations between MaReS and the REFLECT rubric were positive (first run: r=0.92 (p<0.001); second run: r=0.29 (p=0.37)).

CONCLUSION

MaReS might be a useful tool to guide students' reflective writing and provide structured feedback in health professions education. Using more essays for a rater training and more training cycles are likely to result in higher IRRs.