Evaluation of medical students--repeatability of marking. Agreement between two markers of short-answer questions.

One year after marker A had marked a series of short-answer questions, marker B re-marked them using the same mark schedule. In only 26 out of 160 cases (16%) did the two markers agree on the final mark. When these 26 papers were analysed in more detail, it was found that the markers agreed on the mark allocation for each question in only 12 cases. The mean marks allocated by the markers were statistically significantly different, and their disagreement affected students even at the extremes of the class where decisions about 'first-class passes' and failures are made.