Measurement plus judgement in examinations.

All examinations require the exercise of judgement at many points, whatever the method of scoring. In the Part 1 Examination of the Royal Australasian College of Surgeons these judgements are inextricably linked with policies on what is being tested (basic science or basic surgery, knowledge or application), what are the needs of practice and what must be covered in the discipline (choices of inclusion/exclusion on breadth/depth, core material/growing edge), and what are the levels of expectation (difficulty of questions, pass level). Examination scores have a measurable reliability. Confidence limits can be calculated on each candidate's score, but decisions must be made in the grey zone around the pass mark. If candidates did not enter the examination with equal chances because of external circumstances, judgements must be made about the level of disadvantage before deciding how to weight their score.