N.M. Bajwa is residency program director, Department of General Pediatrics, Children's Hospital, Geneva University Hospitals, and faculty member, Unit of Development and Research in Medical Education, Faculty of Medicine, University of Geneva, Geneva, Switzerland; ORCID: http://orcid.org/0000-0002-1445-4594. M.R. Nendaz is professor and director, Unit of Development and Research in Medical Education, Faculty of Medicine, University of Geneva, and attending physician, Division of General Internal Medicine, Geneva University Hospitals, Geneva, Switzerland; ORCID: http://orcid.org/0000-0003-3795-3254. A. Galetto-Lacour is professor and pediatric clerkship director, Department of Pediatric Emergency Medicine, Children's Hospital, Geneva University Hospitals, Geneva, Switzerland; ORCID: https://orcid.org/0000-0002-7901-1647. K. Posfay-Barbe is professor and chairperson, Department of General Pediatrics, Children's Hospital, Geneva University Hospitals, Geneva, Switzerland; ORCID: https://orcid.org/0000-0001-9464-5704. R. Yudkowsky is professor, Department of Medical Education, College of Medicine, University of Illinois at Chicago, Chicago, Illinois; ORCID: https://orcid.org/0000-0002-2145-7582. Y.S. Park is associate professor, Department of Medical Education, College of Medicine, University of Illinois at Chicago, Chicago, Illinois; ORCID: http://orcid.org/0000-0001-8583-4335.
Acad Med. 2019 Nov;94(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 58th Annual Research in Medical Education Sessions):S57-S63. doi: 10.1097/ACM.0000000000002895.
The residency admissions process is a high-stakes assessment system with the purpose of identifying applicants who best meet standards of the residency program and the medical specialty. Prior studies have found that professionalism issues contribute significantly to residents in difficulty during training. This study examines the reliability (internal structure) and predictive (relations to other variables) validity evidence for a standardized patient (SP)-based Professionalism Mini-Evaluation Exercise (P-MEX) using longitudinal data from pediatrics candidates from admission to the end of the first year of postgraduate training.
Data from 5 cohorts from 2012 to 2016 (195 invited applicants) were analyzed from the University of Geneva (Switzerland) Pediatrics Residency Program. Generalizability theory was used to examine the reliability and variance components of the P-MEX scores, gathered across 3 cases. Correlations and mixed-effects regression analyses were used to examine the predictive utility of SP-based P-MEX scores (gathered as part of the admissions process) with rotation evaluation scores (obtained during the first year of residency).
Generalizability was moderate (G coefficient = 0.52). Regression analyses predicting P-MEX scores to first-year rotation evaluations indicated significant standardized effect sizes for attitude and personality (β = 0.36, P = .02), global evaluation (β = 0.27, P = .048), and total evaluation scores (β = 0.34, P = .04).
Validity evidence supports the use of P-MEX scores as part of the admissions process to assess professionalism. P-MEX scores provide a snapshot of an applicant's level of professionalism and may predict performance during the first year of residency.
住院医师招生过程是一个高风险的评估系统，其目的是识别出最符合住院医师计划和医学专业标准的申请人。先前的研究发现，专业精神问题对培训期间有困难的住院医师有重大影响。本研究使用来自日内瓦大学（瑞士）儿科学住院医师培训计划的 2012 年至 2016 年的 5 个队列的纵向数据，检验基于标准化患者（SP）的专业精神微型评估练习（P-MEX）的可靠性（内部结构）和预测（与其他变量的关系）有效性证据。
分析了来自日内瓦大学儿科学住院医师培训计划的 5 个队列（2012 年至 2016 年，共 195 名受邀申请人）的数据。使用广义理论来检验 P-MEX 分数的可靠性和方差分量，这些分数是在 3 个案例中收集的。相关性和混合效应回归分析用于检验 SP 基于 P-MEX 分数（作为招生过程的一部分收集）与第一年住院医师轮转评估分数（在住院医师第一年获得）的预测效用。
可推广性中等（G 系数=0.52）。预测 P-MEX 分数与第一年轮转评估的回归分析表明，态度和人格（β=0.36，P=0.02）、总体评价（β=0.27，P=0.048）和总评价分数（β=0.34，P=0.04）的标准化效应大小具有统计学意义。
有效性证据支持将 P-MEX 分数用作招生过程的一部分，以评估专业精神。P-MEX 分数提供了申请人专业精神水平的快照，并可能预测住院医师第一年的表现。
