Texas Tech University Health Sciences School of Pharmacy, Department of Pharmacy Practice, Amarillo.
J Multidiscip Healthc. 2010 Sep 9;3:169-79. doi: 10.2147/JMDH.S12346.
Formal presentations are a common requirement for students in health professional programs, and evaluations are often viewed as subjective. To date, literature describing the reliability or validity of seminar grading rubrics is lacking. The objectives of this study were to characterize inter-rater agreement and internal consistency of a grading rubric used in a grand rounds seminar course.
Retrospective study of 252 student presentations given from fall 2007 to fall 2008. Data including student and faculty demographics, overall content score, overall communication scores, subcomponents of content and communication, and total presentation scores were collected. Statistical analyses were performed using SPSS, 16.0.
The rubric demonstrated internal consistency (Cronbach's alpha = 0.826). Mean grade difference between faculty graders was 4.54 percentage points (SD = 3.614), with ≤ 10-point difference for 92.5% of faculty evaluations. Student self evaluations correlated with faculty scores for content, communication, and overall presentation (r = 0.513, r = 0.455, and r = 0.539; P < 0.001 for all respectively). When comparing mean faculty scores to student's self-evaluations between quintiles, students with lower faculty evaluations overestimated their performance, and those with high faculty evaluations underestimated their performance (P < 0.001).
The seminar evaluation rubric demonstrated inter-rater agreement and internal consistency.
正式演讲是健康专业学生的常见要求，而评估通常被视为主观的。迄今为止，描述研讨会评分细则的可靠性或有效性的文献还很少。本研究的目的是描述在一个大巡回研讨会课程中使用的评分细则的评分者间一致性和内部一致性。
对 2007 年秋季至 2008 年秋季期间的 252 名学生演讲进行回顾性研究。收集的数据包括学生和教师的人口统计学信息、总体内容评分、总体沟通评分、内容和沟通的子组件以及总体演示评分。使用 SPSS 16.0 进行统计分析。
该评分细则表现出内部一致性（Cronbach's alpha = 0.826）。教师评分之间的平均分数差异为 4.54 个百分点（SD = 3.614），92.5%的教师评估分数差异在 10 分以内。学生自我评估与教师对内容、沟通和总体演示的评分相关（r = 0.513，r = 0.455，r = 0.539；分别为 P < 0.001）。当将平均教师评分与学生在五分位数之间的自我评估进行比较时，教师评分较低的学生高估了自己的表现，而教师评分较高的学生低估了自己的表现（均为 P < 0.001）。
研讨会评估评分细则表现出评分者间一致性和内部一致性。
