Department of Biochemistry and Molecular Cell Biology, University Medical Center Hamburg-Eppendorf, N30, Martinistraße 52, 20246, Hamburg, Germany.
Department of Psychology, University of Münster, Münster, Germany.
BMC Med Educ. 2022 Apr 20;22(1):293. doi: 10.1186/s12909-022-03305-x.
Social skills are important for future physicians and are therefore increasingly considered in selection processes. One economic assessment method from which different social skills can be inferred are Situational Judgment Tests (SJTs) in which applicants are asked to rate behavioral responses in context-relevant situations. However, traditional SJTs have so far failed to distinctively measure specified constructs. To address this shortcoming in the medical admission context, we applied a construct-driven approach of SJT development in which test development was deductively guided by agency and communion as target constructs.
The final version of the construct-driven SJT includes 15 items per construct with three behavioral responses. Medical school applicants (N = 1527) completed the construct-driven SJT, a traditional SJT, and an aptitude test under high-stakes condition as part of their application. A subsample (N = 575) participated in a subsequent voluntary online study with self-report measures of personality and past behavior.
The proposed two-factor structure and internal consistency of the construct-driven SJT was confirmed. Communal SJT scores were positively associated with self-reported communal personality and communal behavior, yet effects were smaller than expected. Findings for agentic SJT scores were mixed with positive small associations to self-reported agentic personality scores and agentic behavior but unexpected negative relations to communal self-reported measures.
Results suggest that construct-driven SJTs might overcome validity limitations of traditional SJTs, although their implementation is challenging. Despite first indicators of validity, future research needs to address practical points of application in high-stakes settings, inclusion of other constructs, and especially prediction of actual behavior before the application of construct-driven SJTs for selection purposes in medical admission can be recommended.
社交技能对未来的医生很重要，因此越来越多地在选拔过程中被考虑。可以推断出不同社交技能的一种经济评估方法是情境判断测验（SJTs），其中要求申请人根据相关情境评估行为反应。然而，传统的 SJTs 迄今为止未能明显测量指定的结构。为了解决医学入学背景下的这一缺点，我们应用了一种情境判断测验的结构驱动方法，其中测验的开发是由机构和交流作为目标结构进行演绎指导的。
结构驱动的 SJTs 的最终版本包括每个结构的 15 个项目，每个项目有三个行为反应。医学院申请者（N=1527）在高风险条件下完成了结构驱动的 SJTs、传统的 SJTs 和一项能力倾向测试，作为申请的一部分。一个子样本（N=575）参与了随后的自愿在线研究，包括人格和过去行为的自我报告测量。
提出的两因素结构和结构驱动 SJTs 的内部一致性得到了确认。社会 SJTs 分数与自我报告的社会人格和社会行为呈正相关，但效果比预期的要小。对代理 SJTs 分数的发现则比较复杂，与自我报告的代理人格分数和代理行为呈正相关，但与自我报告的社会分数呈意外的负相关。
结果表明，结构驱动的 SJTs 可能克服了传统 SJTs 的有效性限制，尽管它们的实施具有挑战性。尽管有了初步的有效性指标，但未来的研究需要解决高风险环境下的实际应用问题，包括其他结构的纳入，特别是在推荐将结构驱动的 SJTs 用于医学入学选拔目的之前，需要预测实际行为。
