Validation of a large-scale clinical examination for international medical graduates.

OBJECTIVE

To evaluate a new examination process for international medical graduates (IMGs) to ensure that it is able to reliably assign candidates to 1 of 4 competency levels, and to determine if a global rating scale can accurately stratify examinees into 4 levels of learners: clerks, first-year residents, second-year residents, or practice ready.

DESIGN

Validation study evaluating a 12-station objective structured clinical examination.

SETTING

Ontario.

PARTICIPANTS

A total of 846 IMGs, and an additional 63 randomly selected volunteers from 2 groups: third-year clinical clerks (n = 42) and first-year family medicine residents (n = 21).

MAIN OUTCOME MEASURES

The accuracy of the stratification of the examinees into learner levels, the impact of the patient-encounter ratings and postencounter oral questions, and between-group differences in total score.

RESULTS

Reliability of the patient-encounter scores, postencounter oral question scores, and the total between-group difference scores was 0.93, 0.88, and 0.76, respectively. Third-year clerks scored the lowest, followed by the IMGs. First-year residents scored highest for all 3 scores. Analysis of variance demonstrated significant between-group differences for all 3 scores (P < .05). Postencounter oral question scores differentiated among all 3 groups.

CONCLUSION

Clinical examination scores were capable of differentiating among the 3 groups. As a group, the IMGs seemed to be less competent than the first-year family medicine residents and more competent than the third-year clerks. The scores generated by the postencounter oral questions were the most effective in differentiating between the 2 training levels and among the 3 groups of test takers.