Pediatric resident performance. The reliability and validity of rating forms.

A large pediatric residency program conducted an extensive analysis of the reliability and validity of the rating forms used to evaluate the pediatric residents enrolled in the program. Analyses were conducted on groups of residents who took the Pediatric In-Training Program (PITE) from 1977 through 1981. An average of 7.1 faculty members rated each resident on a standard form. The data indicate that although the reliability of individual ratings is very low, several factors achieved acceptable levels of reliability when aggregated. The first-year rating of history-taking ability correlated significantly with the PITE but ratings for more advanced residents did not correlate with PITE scores, mostly because of ceiling effects.