Stern David T, Friedman Ben-David Miriam, Norcini John, Wojtczak Andrzej, Schwarz M Roy
Department of Internal Medicine , University of Michigan Medical School, Ann Arbor, Michigan, USA.
Med Educ. 2006 Feb;40(2):166-72. doi: 10.1111/j.1365-2929.2005.02374.x.
To establish international standards for medical schools, an appropriate panel of experts must decide on performance standards. A pilot test of such standards was set in the context of a multidimensional (multiple-choice question examination, objective structured clinical examination, faculty observation) examination at 8 leading schools in China.
A group of 16 medical education leaders from a broad array of countries met over a 3-day period. These individuals considered competency domains, examination items, and the percentage of students who could fall below a cut-off score if the school was still to be considered as meeting competencies. This 2-step process started with a discussion of the borderline school and the relative difficulty of a borderline school in achieving acceptable standards in a given competency domain. Committee members then estimated the percentage of students falling below the standard that is tolerable at a borderline school and were allowed to revise their ratings after viewing pilot data.
Tolerable failure rates ranged from 10% to 26% across competency domains and examination types. As with other standard-setting exercises, standard deviations from initial to final estimates of the tolerable failure rates fell, but the cut-off scores did not change significantly. Final, but not initial cut-off scores were correlated with student failure rates (r = 0.59, P = 0.03).
This paper describes a method to set school-level outcome standards at an international level based on prior established standard-setting methods. Further refinement of this process and validation using other examinations in other countries will be needed to achieve accurate international standards.
为制定医学院校的国际标准，一个合适的专家小组必须确定绩效标准。在中国8所顶尖院校进行的多维度（多项选择题考试、客观结构化临床考试、教师观察）考试的背景下，对这些标准进行了试点测试。
来自众多国家的16位医学教育领导者组成的小组进行了为期3天的会议。这些人审议了能力领域、考试项目，以及如果学校仍被视为符合能力要求，可能低于及格分数的学生比例。这个两步过程首先讨论了临界学校以及临界学校在给定能力领域达到可接受标准的相对难度。委员会成员随后估计了临界学校可容忍的低于标准的学生比例，并在查看试点数据后被允许修改他们的评分。
各能力领域和考试类型的可容忍不及格率在10%至26%之间。与其他标准设定活动一样，可容忍不及格率的初始估计值到最终估计值的标准差下降了，但及格分数没有显著变化。最终及格分数（而非初始及格分数）与学生不及格率相关（r = 0.59，P = 0.03）。
本文描述了一种基于先前既定的标准设定方法在国际层面设定学校层面结果标准的方法。要实现准确的国际标准，需要对这一过程进行进一步完善，并在其他国家使用其他考试进行验证。
