Schmitt H, Robra B P
Institut für Sozialmedizin und Gesundheitsökonomie, Magdeburg, Germany.
Dtsch Med Wochenschr. 2002 Feb 1;127(5):193-8. doi: 10.1055/s-2002-19905.
Teaching evaluations should create an incentive to improve academic teaching quality. Teaching also gains importance in the performance-oriented allocation of resources. The results of the country wide written examinations conducted by the Federal Institute for Medical and Pharmaceutical Examination Questions (IMPP) are obvious indicators for purposes of comparative evaluation. However, a comparison of medical faculties is confounded by heterogeneous proportions of students in the so called "reference group" This group includes students who attend their examinations within the regular duration of study; these candidates achieve significantly better results than other students. This contribution describes a comparative educational achievement index (aJaX) adjusting for the varying proportions of students in the reference group.
The IMPP publishes the results of the examinations, disaggregated according to faculties, type of examination, examination subjects, examination semester and membership in the reference group. Aggregation of the semi-annual data to annual data leads to equal treatment of faculties with different admittance regulations. The proportion of students in the reference group is taken into account by calculating an adjusted annual index (aJaX) which compares the observed result of the respective medical faculty with the result one would expect if the average (federal) outcome would apply to a faculty with an equal proportion of students in the reference group (i. e. indirect standardisation).
The success rates of the faculties, as defined by the adjusted annual index aJaX, lie with a few exceptions in the interval between 0.95 and 1.05 during the years 1994 to 2000. The results of the second year examinations demonstrate the largest variance. Over time strong deviations rarely occur. These stable results are not observed with the frequently used reference-autumn-index (ReX), which only takes into consideration the results of the reference groups in the autumn examinations. Temporal trends therefore become more clearly visible in the aJaX. The faculties of the new federal states show a clear upward trend in the analysed period.
The results of the medical faculties in the uniform examinations can be compared using an adjusted annual index aJaX. Advantages of the aJaX lie in its stability and neutrality, which are obtained by the annual aggregation of data and the reference group adjustment. The low degree of scatter emphasises a high standardisation in academic medical teaching. However, the aJaX measures only one output of teaching. For the assessment of teaching efficiency in the sense of an input/output relation, inputs must also be taken into account, such as the final school grades of the entering and transferring students.
教学评估应激励学术教学质量的提高。在以绩效为导向的资源分配中，教学也变得愈发重要。联邦医学与药学试题研究所（IMPP）组织的全国性笔试结果是进行比较评估的明显指标。然而，医学院校之间的比较因所谓“参考组”中学生比例的差异而变得复杂。该组包括在正常学习期限内参加考试的学生；这些考生的成绩明显优于其他学生。本文介绍了一种比较教育成就指数（aJaX），用于调整参考组中学生比例的差异。
IMPP公布考试结果，按学院、考试类型、考试科目、考试学期以及参考组成员进行分类。将半年数据汇总为年度数据，可使具有不同入学规定的学院得到平等对待。通过计算调整后的年度指数（aJaX）来考虑参考组中学生的比例，该指数将各医学院的实际观察结果与假设平均（联邦）成绩适用于参考组学生比例相同的学院时所预期的结果进行比较（即间接标准化）。
根据调整后的年度指数aJaX定义，各学院的成功率在1994年至2000年期间，除少数情况外，处于0.95至1.05之间。二年级考试的结果差异最大。随着时间推移，很少出现强烈偏差。常用的参考秋季指数（ReX）仅考虑秋季考试中参考组的结果，未观察到这种稳定结果。因此，在aJaX中时间趋势更清晰可见。在分析期内，新联邦州的学院呈现出明显的上升趋势。
可使用调整后的年度指数aJaX对医学院校在统一考试中的结果进行比较。aJaX的优点在于其稳定性和中立性，这通过年度数据汇总和参考组调整获得。低离散度强调了学术医学教学的高度标准化。然而，aJaX仅衡量教学的一个产出。要从投入/产出关系的角度评估教学效率，还必须考虑投入因素，如入学和转学学生的高中毕业成绩。
