Hecker Kent, Violato Claudio
Health Sciences Centre, Department of Veterinary Clinical and Diagnostic Sciece, Faculty of Veterinary Medicine, University of Calgary, Calgary, AB, Canada.
J Vet Med Educ. 2010 Fall;37(3):254-7. doi: 10.3138/jvme.37.3.254.
The reliability and validity of using essays for veterinary medical school admissions requires investigation. We explored the reliability and construct validity of a structured essay station in the 2009 admission process at the University of Calgary Faculty of Veterinary Medicine. One hundred three applicants (female=80.6%, male=19.4%; mean age=23.05 years, SD=3.96) participated. Each applicant wrote a one-hour supervised essay (750 words). Essays were rated independently by two randomly assigned raters (n=16). Raters scored essays on four items, each on a five-point anchored scale. Nine essays were scored by all raters to perform a decision study. Generalizability analysis resulted in a reliability coefficient of 0.55. The decision study indicated that three raters and four items produces a G of 0.68. Essay score correlated with interview score (r=0.30, p<0.01) but not with GPA (r=0.05, p=ns). Overall reliability was adequate and higher than what has been reported for unsupervised written submissions. Results from the decision study suggest that three raters with four items provide adequate reliability. Correlations with interviews and grade point average provide evidence of construct validity. A time-limited essay with a clear scoring protocol results in adequate reliability and some validity.
使用文章作为兽医学院入学申请材料的可靠性和有效性需要进行调查。我们探讨了卡尔加里大学兽医学院2009年招生过程中一个结构化文章环节的可靠性和结构效度。103名申请者（女性占80.6%，男性占19.4%；平均年龄23.05岁，标准差3.96）参与其中。每位申请者写一篇一小时的命题文章（750字），由两名随机分配的评分者独立评分（共16名评分者）。评分者根据四个项目对文章进行评分，每个项目采用五分制。所有评分者对9篇文章进行评分以进行决策研究。概化分析得出信度系数为0.55。决策研究表明，三名评分者和四个项目得出的G值为0.68。文章分数与面试分数相关（r = 0.30，p < 0.01），但与平均绩点无关（r = 0.05，p =无显著性差异）。总体信度足够，高于无监督书面材料所报告的信度。决策研究结果表明，三名评分者和四个项目能提供足够的信度。与面试和平均绩点的相关性提供了结构效度的证据。限时文章加上明确的评分方案能产生足够的信度和一定的效度。
