How to evaluate intraexaminer reliability using an interexaminer reliability study design.

Examiner reliability is often investigated to make generalizations about a profession's performance with various diagnostic tests. Although the evaluation of interexaminer reliability is straightforward, assessment of intraexaminer reliability can be problematic. Estimations of intrarater reliability can be inflated due to correlated error and the difficulty in blinding the examiners. A statistical method is presented that permits the investigator to compute intraexaminer as well as interexaminer reliability and precision from the findings of an interexaminer reliability investigation. In this approach, intraclass correlation coefficients are constructed from variance components estimated from a simple repeated-measures design: each of two or more raters evaluates each subject one time. The method can be applied to nominal and ordinal as well as interval data. Not only can spurious results be avoided, but time and funds can be saved by reducing the required number of subject ratings.