Smith R M
University of Hawaii at Manoa, John A. Burns School of Medicine, Honolulu 96822.
Acad Med. 1993 May;68(5):366-72. doi: 10.1097/00001888-199305000-00020.
The three-step triple jump (TJ) examination aims to assess students' clinical problem-solving processes predominantly by means of subjective assessments administered by faculty. But training TJ administrators to ensure interrater reliability is both time- and cost-intensive, and difficult at best--hence the desire to test a more objective system of scoring students' TJ performances.
The sample was the 58 first-year students of the class of 1995, who in March 1992 were finishing the second 13-week unit in the problem-based curriculum at the University of Hawaii John A. Burns School of Medicine. To determine how well the school had succeeded in standardizing and objectifying its TJ examination (used for all units in the first two years), scores were correlated for various objective examinations independent of the TJ, the TJ administrators' subjective assessments, and the TJ objective assessments (regarding number of problem-based hypotheses generated and number of hypothesis-testing clinical database items elicited). The statistical methods used were linear regression, Student's unpaired t-test, chi-square, and the z-test.
The TJ scores and independent objective examinations did not correlate significantly (suggesting that they assess different aspects of student achievement), but the TJ subjective and objective scores did correlate significantly. There were large standard deviations on the TJ objective scores, largely because one problem was significantly more difficult than the others (each student works on only one of several problems). However, the administrators' subjective scores for all problems were comparable.
Because problems vary in difficulty, objective scores cannot be used across problems as a major component of all students' grades; but when a student has received an unsatisfactory score, an external reviewer can evaluate the appropriateness of the subjective score by comparing the student's objective performance with those of students who had the same problem and received higher subjective scores. That the administrators' subjective assessments for all problems were comparable not only suggests that the administrators were able to adjust for problem variability but also reinforces the appropriateness of using subjective assessments for the TJ examination.
三步三级跳远（TJ）考试主要通过教师进行的主观评估来评估学生的临床问题解决过程。但是培训TJ评估人员以确保评分者间的可靠性既耗时又成本高昂，而且充其量也很困难——因此人们希望测试一种更客观的学生TJ表现评分系统。
样本为1995级的58名一年级学生，他们于1992年3月在夏威夷大学约翰·A·伯恩斯医学院以问题为基础的课程中完成了第二个为期13周的单元。为了确定该学校在标准化和客观化其TJ考试（用于前两年的所有单元）方面取得了多大成功，将与TJ无关的各种客观考试成绩、TJ评估人员的主观评估以及TJ客观评估（关于生成的基于问题的假设数量和引出的假设检验临床数据库项目数量）进行了相关性分析。所使用的统计方法包括线性回归、学生未配对t检验、卡方检验和z检验。
TJ成绩与独立客观考试成绩之间没有显著相关性（表明它们评估了学生成绩的不同方面），但TJ主观成绩与客观成绩之间存在显著相关性。TJ客观成绩的标准差很大，主要是因为有一个问题比其他问题明显更难（每个学生只处理几个问题中的一个）。然而，评估人员对所有问题的主观评分是可比的。
由于问题难度不同，客观成绩不能作为所有学生成绩的主要组成部分在不同问题间通用；但是当学生得到不满意的成绩时，外部评审人员可以通过将该学生的客观表现与处理相同问题且得到更高主观分数的学生的表现进行比较，来评估主观分数的合理性。评估人员对所有问题的主观评估具有可比性，这不仅表明评估人员能够针对问题的变异性进行调整，也强化了在TJ考试中使用主观评估的合理性。
