Replication of an Intervention to Mitigate Gender Bias in Student Evaluations of Teaching Yields Variable Results Across a Biology Department.

Student evaluations of teaching (SET) have repeatedly been shown to be biased against women instructors. Although few have been able to mitigate these biases, one team reported success in two courses by adding a short AntiBias statement to the beginning of SETs. We conducted a conceptual replication of that study to investigate the effectiveness of the AntiBias statement across a Department of Biological Sciences over three semesters. The AntiBias treatment inconsistently affected the SETs, sometimes improving women's scores but often not having any effect. Qualitative analysis showed that the types of comments students gave were mostly not affected by the conditions of treatment or instructor gender and were most frequently framed in positive connotation, implicitly about the instructor, and about course characteristics such as the logistics of the course. Our findings do not support the consistent replicability of the original work scaled to the department level yet shine an important light on SETs in the biology context. Moreover, this work suggests that a simple intervention to mitigate gender bias in teaching evaluations is not sufficient to remedy the multitude of issues with SETs. We discuss differences among studies and suggestions from the literature on ways to improve the evaluation of teaching.