Evaluation using simulated students for reliability of multiple standardized-patients scoring in clinical performance examinations.

BACKGROUND

The consistency of rating among different standardized patients (SPs) is very important in clinical performance examinations (CPX).

AIMS

We evaluated the usefulness of simulated students to assess the reliability of a group of standardized patients in a CPX.

METHODS

Five SPs in each case were trained to simulate the same patient in a CPX. Ten fourth-year medical student volunteers were selected to act as simulated students. After the simulated students encountered the first SP, they reviewed a video of their performance and sequentially repeated the performance with the other 4 SPs.

RESULTS

The average maximum difference (Deltamax +/- SD) of multiple SPs' percent scores on a simulated student was 18 +/- 7. In 10 pairs of SPs who gave largest Deltamax on a given student, the median Deltamax was 30 (range, 26-38) while median difference of percent score in the faculty scoring on the same simulated student was 3 (range, 0-7). Five problematic SPs whose ratings were significantly different from 3 or more other SPs' ratings in each case were identified.

CONCLUSIONS

Simulated students may be a useful means to check the consistency of rating among different SPs in a CPX.