[The "Adjusted Annual Index" (aJaX): A proposal for the use of the IMPP results in the evaluation of teaching at German medical faculties].

BACKGROUND AND OBJECTIVE

Teaching evaluations should create an incentive to improve academic teaching quality. Teaching also gains importance in the performance-oriented allocation of resources. The results of the country wide written examinations conducted by the Federal Institute for Medical and Pharmaceutical Examination Questions (IMPP) are obvious indicators for purposes of comparative evaluation. However, a comparison of medical faculties is confounded by heterogeneous proportions of students in the so called "reference group" This group includes students who attend their examinations within the regular duration of study; these candidates achieve significantly better results than other students. This contribution describes a comparative educational achievement index (aJaX) adjusting for the varying proportions of students in the reference group.

MATERIAL AND METHODS

The IMPP publishes the results of the examinations, disaggregated according to faculties, type of examination, examination subjects, examination semester and membership in the reference group. Aggregation of the semi-annual data to annual data leads to equal treatment of faculties with different admittance regulations. The proportion of students in the reference group is taken into account by calculating an adjusted annual index (aJaX) which compares the observed result of the respective medical faculty with the result one would expect if the average (federal) outcome would apply to a faculty with an equal proportion of students in the reference group (i. e. indirect standardisation).

RESULTS

The success rates of the faculties, as defined by the adjusted annual index aJaX, lie with a few exceptions in the interval between 0.95 and 1.05 during the years 1994 to 2000. The results of the second year examinations demonstrate the largest variance. Over time strong deviations rarely occur. These stable results are not observed with the frequently used reference-autumn-index (ReX), which only takes into consideration the results of the reference groups in the autumn examinations. Temporal trends therefore become more clearly visible in the aJaX. The faculties of the new federal states show a clear upward trend in the analysed period.

CONCLUSIONS

The results of the medical faculties in the uniform examinations can be compared using an adjusted annual index aJaX. Advantages of the aJaX lie in its stability and neutrality, which are obtained by the annual aggregation of data and the reference group adjustment. The low degree of scatter emphasises a high standardisation in academic medical teaching. However, the aJaX measures only one output of teaching. For the assessment of teaching efficiency in the sense of an input/output relation, inputs must also be taken into account, such as the final school grades of the entering and transferring students.