Jackson Jeffrey L, Gavinski Katherine, Thompson Michelle G, Storch Derek, Murphy Mary G, Nickoloff Sarah, Kuriyama Akira
Clement J Zablocki VAMC, 5000 W National Ave, Milwaukee, WI, USA.
Department of Primary Care and Emergency Medicine, Kyoto University Graduate School of Medicine, Kyoto, Japan.
J Gen Intern Med. 2025 Apr;40(5):996-1002. doi: 10.1007/s11606-024-09085-y. Epub 2025 Jan 8.
Our purpose was to evaluate the characteristics of highly and poorly rated teachers as well as to assess the validity and reliability of those evaluations.
We downloaded 14 years of medicine faculty evaluations completed by 3rd and 4 year medical students. We dichotomized overall teaching effectiveness as outstanding (receiving "outstanding") or inferior (rated as a "unsatisfactory," "marginal," or "acceptable"). We analyzed these using logistic regression (STATA v 18.0). We assessed validity and reliability using factor analysis, Cronbach's alpha, and intraclass correlation coefficients.
Most (57%) of the 722 faculty members were rated as outstanding. Medical students valued faculty that took advantage of opportunities to teach (OR, 3.0; 95% CI, 2.7-3.3), who were enthusiastic (OR, 2.6; 95% CI, 2.3-2.9), and clear/organized (OR, 2.5; 2.3-2.7). Faculty rarely were rated as inferior (7.7%). Among lower-rated faculty, 91% had more than one lower evaluation. Lower-rated teachers had lower ratings on most domains of evaluation including taking advantages of opportunities to teach (4.6 vs. 2.7, p < 0.0005), being clear and organized (3.0 vs. 4.6, p < 0.0005), enthusiasm (4.5 vs. 2.7, p < 0.0005), being supportive (4.5 vs. 2.5, p < 0.0005), providing feedback (4.4 vs. 2.6, p < 0.005), or clearly answering questions (4.6 vs. 3.1, p < 0.0005). While evaluations were highly consistent (Cronbach's alpha, 0.94), there were low levels of agreement with intraclass correlation coefficients ranging from 0.09 to 0.36.
Most attendings received high ratings, while lower ratings were uncommon. Most teachers receiving lower ratings received more than one, suggesting that lower ratings may be a better discriminator of teaching effectiveness than outstanding ones. Teaching ratings had low inter-rater reliability, suggesting either low validity or that learners value different characteristics in teachers.
我们的目的是评估评分高和评分低的教师的特征，并评估这些评价的有效性和可靠性。
我们下载了14年由三年级和四年级医学生完成的医学教师评价。我们将总体教学效果分为优秀（获得“优秀”评价）或较差（评为“不满意”“勉强合格”或“合格”）。我们使用逻辑回归（STATA v 18.0）对这些数据进行分析。我们使用因子分析、克朗巴哈系数和组内相关系数来评估有效性和可靠性。
722名教师中大多数（57%）被评为优秀。医学生重视那些利用教学机会的教师（比值比[OR]，3.0；95%置信区间[CI]，2.7 - 3.3）、热情的教师（OR，2.6；95% CI，2.3 - 2.9）以及条理清晰的教师（OR，2.5；2.3 - 2.7）。教师很少被评为较差（7.7%）。在评分较低的教师中，91%有不止一项较低评价。评分较低的教师在大多数评价领域的评分较低，包括利用教学机会（4.6对2.7，p < 0.0005）、条理清晰（3.0对4.6，p < 0.0005）、热情（4.5对2.7，p < 0.0005）、给予支持（4.5对2.5，p < 0.0005）、提供反馈（4.4对2.6，p < 0.005）或清晰回答问题（4.6对3.1，p < 0.0005）。虽然评价具有高度一致性（克朗巴哈系数，0.94），但组内相关系数的一致性水平较低，范围从0.09到0.36。
大多数主治医师获得高评分，而低评分并不常见。大多数获得低评分的教师有不止一项低评分，这表明低评分可能比高评分更能区分教学效果。教学评分的评分者间信度较低，这表明要么有效性较低，要么学习者重视教师的不同特征。
