Acuity Insights, Toronto, Canada.
Deloitte, Toronto, Canada.
PLoS One. 2023 Jan 26;18(1):e0280493. doi: 10.1371/journal.pone.0280493. eCollection 2023.
Situational judgments tests have been increasingly used to help training programs for the health professions incorporate professionalism attributes into their admissions process. While such tests have strong psychometric properties for testing professional attributes and are feasible to implement in high-volume, high-stakes selection, little is known about constructed-response situational judgment tests and their validity.
We will conduct a systematic review of primary published or unpublished studies reporting on the association between scores on constructed-response situational judgment tests and scores on other tests that measure personal, interpersonal, or professional attributes in training programs for the health professions. In addition to searching electronic databases, we will contact academics and researchers and undertake backward and forward searching. Two reviewers will independently screen the papers and decide on their inclusion, first based on the titles and abstracts of all citations, and then according to the full texts. Data extraction will be done independently by two reviewers using a data extraction form to chart study details and key findings. Studies will be assessed for the risk of bias and quality by two reviewers using the "Quality In Prognosis Studies" tool. To synthesize evidence, we will test the statistical heterogeneity and conduct a psychometric meta-analysis using a random-effects model. If adequate data are available, we will explore whether the meta-analytic correlation varies across different subgroups (e.g., race, gender).
The findings of this study will inform best practices for admission and selection of applicants for training programs for the health professions and encourage further research on constructed-response situational judgment tests, in particular their validity.
The protocol for this systematic review has been registered in PROSPERO [CRD42022314561]. https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022314561.
情境判断测试已越来越多地被用于帮助医疗专业的培训项目将专业精神属性纳入其招生过程。虽然此类测试在测试专业属性方面具有较强的心理测量学特性，并且在大规模、高风险选拔中具有可行性，但对于构建反应情境判断测试及其有效性的了解甚少。
我们将对主要的已发表或未发表的研究进行系统评价，这些研究报告了构建反应情境判断测试的分数与其他测试的分数之间的关联，这些测试用于衡量医疗专业培训项目中的个人、人际或专业属性。除了搜索电子数据库外，我们还将联系学者和研究人员，并进行回溯和向前搜索。两位评审员将独立筛选论文，并根据所有引文的标题和摘要首先决定其是否包含，然后根据全文决定。两位评审员将使用数据提取表独立提取数据，以绘制研究细节和主要发现。两位评审员将使用“预后研究质量工具”评估研究的偏倚风险和质量。为了综合证据，我们将使用随机效应模型检验统计异质性并进行心理测量荟萃分析。如果有足够的数据，我们将探索元分析相关性是否因不同的亚组（例如，种族、性别）而有所不同。
本研究的结果将为医疗专业培训项目的招生和申请人选拔提供最佳实践，并鼓励对构建反应情境判断测试进行进一步研究，特别是其有效性。
本系统评价的方案已在 PROSPERO [CRD42022314561] 中注册。https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022314561。
