Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada.
Med Educ. 2009 Aug;43(8):767-75. doi: 10.1111/j.1365-2923.2009.03407.x.
In this paper we report on further tests of the validity of the multiple mini-interview (MMI) selection process, comparing MMI scores with those achieved on a national high-stakes clinical skills examination. We also continue to explore the stability of candidate performance and the extent to which so-called 'cognitive' and 'non-cognitive' qualities should be deemed independent of one another.
To examine predictive validity, MMI data were matched with licensing examination data for both undergraduate (n = 34) and postgraduate (n = 22) samples of participants. To assess the stability of candidate performance, reliability coefficients were generated for eight distinct samples. Finally, correlations were calculated between 'cognitive' and 'non-cognitive' measures of ability collected in the admissions procedure, on graduation from medical school and 18 months into postgraduate training.
The median reliability of eight administrations of the MMI in various cohorts was 0.73 when 12 10-minute stations were used with one examiner per station. The correlation between performance on the MMI and number of stations passed on an objective structured clinical examination-based licensing examination was r = 0.43 (P < 0.05) in a postgraduate sample and r = 0.35 (P < 0.05) in an undergraduate sample of subjects who sat the MMI 5 years prior to sitting the licensing examination. The correlation between 'cognitive' and 'non-cognitive' assessment instruments increased with time in training (i.e. as the focus of the assessments became more tailored to the clinical practice of medicine).
Further evidence for the validity of the MMI approach to making admissions decisions has been provided. More generally, the reported findings cast further doubt on the extent to which performance can be captured with trait-based models of ability. Finally, although a complementary predictive relationship has consistently been observed between grade point average and MMI results, the extent to which cognitive and non-cognitive qualities are distinct appears to depend on the scope of practice within which the two classes of qualities are assessed.
本文进一步测试了多迷你面试（MMI）选拔过程的有效性，将 MMI 分数与全国高风险临床技能考试的分数进行比较。我们还继续探索候选人表现的稳定性，以及“认知”和“非认知”素质应该在多大程度上被视为彼此独立。
为了检验预测效度，将 MMI 数据与本科（n=34）和研究生（n=22）参与者样本的执照考试数据相匹配。为了评估候选人表现的稳定性，为八个不同的样本生成了可靠性系数。最后，在入学程序中收集的“认知”和“非认知”能力测量之间计算了相关性，并在医学院毕业和研究生培训 18 个月后进行了相关性分析。
当使用 12 个 10 分钟的站点和每个站点一个考官时，在各种队列中，MMI 的中位数可靠性为 0.73。在一个研究生样本中，MMI 表现与基于客观结构化临床考试的执照考试中通过的站点数量之间的相关系数为 r = 0.43（P<0.05），在一个本科样本中，该相关系数为 r = 0.35（P<0.05）。在参加执照考试前 5 年参加 MMI 的受试者中。“认知”和“非认知”评估工具之间的相关性随着培训时间的增加而增加（即评估的重点越来越符合医学实践）。
提供了进一步证明 MMI 方法在做出招生决策方面有效性的证据。更一般地说，所报告的发现进一步质疑了通过基于特质的能力模型捕捉表现的程度。最后，尽管 GPA 和 MMI 成绩之间始终存在互补的预测关系，但认知和非认知素质之间的区别程度似乎取决于评估这两类素质的实践范围。
