Department of Biochemistry and Molecular Cell Biology, Center for Experimental Medicine, University Medical Center Hamburg-Eppendorf, Martinistraße 52, Hamburg 20246, Germany.
BMC Med Educ. 2014 Mar 19;14:54. doi: 10.1186/1472-6920-14-54.
Multiple mini-interviews (MMIs) are a valuable tool in medical school selection due to their broad acceptance and promising psychometric properties. With respect to the high expenses associated with this procedure, the discussion about its feasibility should be extended to cost-effectiveness issues.
Following a pilot test of MMIs for medical school admission at Hamburg University in 2009 (HAM-Int), we took several actions to improve reliability and to reduce costs of the subsequent procedure in 2010. For both years, we assessed overall and inter-rater reliabilities based on multilevel analyses. Moreover, we provide a detailed specification of costs, as well as an extrapolation of the interrelation of costs, reliability, and the setup of the procedure.
The overall reliability of the initial 2009 HAM-Int procedure with twelve stations and an average of 2.33 raters per station was ICC=0.75. Following the improvement actions, in 2010 the ICC remained stable at 0.76, despite the reduction of the process to nine stations and 2.17 raters per station. Moreover, costs were cut down from $915 to $495 per candidate. With the 2010 modalities, we could have reached an ICC of 0.80 with 16 single rater stations ($570 per candidate).
With respect to reliability and cost-efficiency, it is generally worthwhile to invest in scoring, rater training and scenario development. Moreover, it is more beneficial to increase the number of stations instead of raters within stations. However, if we want to achieve more than 80 % reliability, a minor improvement is paid with skyrocketing costs.
多站迷你面试（MMI）因其广泛的接受度和有前景的心理测量学特性，成为医学院选择的一种有价值的工具。鉴于该程序的高费用，关于其可行性的讨论应扩展到成本效益问题。
在汉堡大学 2009 年（HAM-Int）进行医学院招生 MMIs 试点测试后，我们采取了多项措施来提高可靠性并降低后续程序的成本。对于这两年，我们都基于多层次分析评估了整体和组内评分者间可靠性。此外，我们提供了成本的详细说明，以及成本、可靠性和程序设置之间关系的外推。
2009 年初始 HAM-Int 程序的总可靠性，有 12 个站，每个站平均有 2.33 名评分者，ICC=0.75。在改进措施之后，2010 年，尽管流程减少到 9 个站，每个站的评分者减少到 2.17 名，但 ICC 仍保持稳定在 0.76。此外，每位考生的成本从 915 美元降至 495 美元。通过 2010 年的模式，我们可以用 16 个单评分者站（每位考生 570 美元）达到 ICC=0.80。
从可靠性和成本效益的角度来看，在评分、评分者培训和情景开发方面进行投资是值得的。此外，增加站的数量而不是每个站的评分者数量更有利。然而，如果我们希望达到 80%以上的可靠性，就需要用飙升的成本来换取微小的改进。
