Bullock Justin L, Seligman Lee, Lai Cindy J, O'Sullivan Patricia S, Hauer Karen E
Department of Medicine, University of California, San Francisco School of Medicine, San Francisco, California, USA.
Department of Medicine, Columbia University Irving Medical Center, New York-Presbyterian Hospital, New York, New York, USA.
Teach Learn Med. 2022 Apr-May;34(2):198-208. doi: 10.1080/10401334.2021.1922285. Epub 2021 May 20.
Clerkship grades contribute to a summative assessment culture in clerkships and can therefore interfere with students' learning. For example, by focusing on summative, tiered clerkship grades, students often discount accompanying feedback that could inform future learning. This case report seeks to explore whether an assessment system intervention which eliminated tiered grades and enhanced feedback was associated with changes in student perceptions of clerkship assessment and perceptions of the clinical learning environment. In January 2019, our institution eliminated tiered clerkship grading (honors/pass/fail) for medical students during the core clerkship year and implemented pass/fail clerkship grading along with required twice weekly, work-based assessments for formative feedback. In this single institution, cross-sectional survey study, we collected data from fourth-year medical students one year after an assessment system intervention. The intervention entailed changing from honors/pass/fail to pass/fail grading in all eight core clerkships and implementing an electronic system to record twice-weekly real-time formative work-based assessments. The survey queried student perceptions on the fairness and accuracy of grading and the clinical learning environment-including whether clerkships were mastery- or performance-oriented. We compared responses from students one year after the assessment intervention to those from the class one year before the intervention. Comparisons were made using unpaired, two-tailed t-tests or chi-squared tests as appropriate with Cohen's d for effect size estimation for score differences. Content analysis was used to analyze responses from two open-ended questions about feedback and grading. Survey response rates were similar before and after intervention (76% (127/168) vs. 72% (118/163), respectively) with no between-group differences in demographics. The after-intervention group showed statistically significant increases in the following factors: "grades are transparent and fair" (Cohen's d = 0.80), "students receive useful feedback" (d = 0.51), and "resident evaluation procedures are fair" (d = 0.40). After-intervention respondents perceived the clerkship learning environment to be more mastery-oriented (d = 0.52), less performance approach-oriented (d = 0.63), and less performance avoid-oriented (d = 0.49). There were no statistical differences in the factors "attending evaluation procedures are fair," "evaluations are accurate," "evaluations are biased," or "perception of stereotype threat." Open-ended questions revealed student recommendations to improve clerkship summary narratives, burden of work-based assessment, and in-person feedback. After an assessment system change to pass/fail grading with work-based assessments, we observed moderate to large improvements in student perceptions of clerkship grading and the mastery orientation of the learning environment. Our intervention did not improve perceptions around bias in assessment in clerkships. Other medical schools may consider similar interventions to begin to address student concerns with clerkship assessment and promote a more adaptive learning environment.
临床实习成绩促成了临床实习中的总结性评估文化,因此可能会干扰学生的学习。例如,通过关注总结性的、分层的临床实习成绩,学生常常忽视那些能为未来学习提供信息的伴随反馈。本病例报告旨在探讨一种评估系统干预措施,即消除分层成绩并加强反馈,是否与学生对临床实习评估的看法以及对临床学习环境的看法的变化相关。2019年1月,我们机构在医学学生核心临床实习年期间取消了分层的临床实习评分(优秀/及格/不及格),实施了及格/不及格的临床实习评分,并要求每周进行两次基于工作的评估以提供形成性反馈。在这项单一机构的横断面调查研究中,我们在评估系统干预一年后收集了四年级医学生的数据。该干预措施包括在所有八门核心临床实习中从优秀/及格/不及格改为及格/不及格评分,并实施一个电子系统来记录每周两次的基于工作的实时形成性评估。该调查询问了学生对评分的公平性和准确性以及临床学习环境的看法,包括临床实习是以掌握为导向还是以表现为导向。我们将评估干预一年后的学生回答与干预前一年的班级学生回答进行了比较。使用未配对的双尾t检验或卡方检验进行比较,并使用科恩d值估计分数差异的效应大小。内容分析用于分析关于反馈和评分的两个开放式问题的回答。干预前后的调查回复率相似(分别为76%(127/168)和72%(118/163)),两组在人口统计学上没有差异。干预后组在以下因素上有统计学意义的增加:“成绩透明且公平”(科恩d = 0.80)、“学生收到有用的反馈”(d = 0.51)以及“住院医师评估程序公平”(d = 0.40)。干预后的受访者认为临床实习学习环境更以掌握为导向(d = 0.52)、较少以成绩导向(d = 0.63)且较少以成绩回避为导向(d = 0.49)。在“主治医生评估程序公平”、“评估准确”、“评估有偏差”或“对刻板印象威胁的感知”等因素上没有统计学差异。开放式问题揭示了学生对改进临床实习总结叙述、基于工作的评估负担和面对面反馈的建议。在将评估系统改为及格/不及格评分并进行基于工作的评估后,我们观察到学生对临床实习评分和学习环境的掌握导向的看法有中度到大幅的改善。我们的干预并没有改善对临床实习评估偏差的看法。其他医学院校可能会考虑类似的干预措施,以开始解决学生对临床实习评估的担忧,并促进一个更具适应性的学习环境。