Institute for Medical Education, University Hospital, LMU Munich, Munich, Germany.
Department of Psychology, Ludwig-Maximilians-Universität München, Munich, Germany.
J Med Internet Res. 2021 Mar 4;23(3):e21196. doi: 10.2196/21196.
Standardized patients (SPs) have been one of the popular assessment methods in clinical teaching for decades, although they are resource intensive. Nowadays, simulated virtual patients (VPs) are increasingly used because they are permanently available and fully scalable to a large audience. However, empirical studies comparing the differential effects of these assessment methods are lacking. Similarly, the relationships between key variables associated with diagnostic competences (ie, diagnostic accuracy and evidence generation) in these assessment methods still require further research.
The aim of this study is to compare perceived authenticity, cognitive load, and diagnostic competences in performance-based assessment using SPs and VPs. This study also aims to examine the relationships of perceived authenticity, cognitive load, and quality of evidence generation with diagnostic accuracy.
We conducted an experimental study with 86 medical students (mean 26.03 years, SD 4.71) focusing on history taking in dyspnea cases. Participants solved three cases with SPs and three cases with VPs in this repeated measures study. After each case, students provided a diagnosis and rated perceived authenticity and cognitive load. The provided diagnosis was scored in terms of diagnostic accuracy; the questions asked by the medical students were rated with respect to their quality of evidence generation. In addition to regular null hypothesis testing, this study used equivalence testing to investigate the absence of meaningful effects.
Perceived authenticity (1-tailed t=11.12; P<.001) was higher for SPs than for VPs. The correlation between diagnostic accuracy and perceived authenticity was very small (r=0.05) and neither equivalent (P=.09) nor statistically significant (P=.32). Cognitive load was equivalent in both assessment methods (t=2.81; P=.003). Intrinsic cognitive load (1-tailed r=-0.30; P=.003) and extraneous load (1-tailed r=-0.29; P=.003) correlated negatively with the combined score for diagnostic accuracy. The quality of evidence generation was positively related to diagnostic accuracy for VPs (1-tailed r=0.38; P<.001); this finding did not hold for SPs (1-tailed r=0.05; P=.32). Comparing both assessment methods with each other, diagnostic accuracy was higher for SPs than for VPs (2-tailed t=2.49; P=.01).
The results on perceived authenticity demonstrate that learners experience SPs as more authentic than VPs. As higher amounts of intrinsic and extraneous cognitive loads are detrimental to performance, both types of cognitive load must be monitored and manipulated systematically in the assessment. Diagnostic accuracy was higher for SPs than for VPs, which could potentially negatively affect students' grades with VPs. We identify and discuss possible reasons for this performance difference between both assessment methods.
标准化患者（SPs）作为一种流行的临床教学评估方法已经存在了几十年，尽管它们需要大量资源。如今，模拟虚拟患者（VPs）越来越多地被使用，因为它们可以永久使用，并可以完全扩展到大量受众。然而，缺乏比较这些评估方法的差异效果的实证研究。同样，与诊断能力相关的关键变量（即诊断准确性和证据生成）之间的关系仍然需要进一步研究。
本研究旨在比较使用 SPs 和 VPs 进行基于表现的评估中的感知真实性、认知负荷和诊断能力。本研究还旨在研究感知真实性、认知负荷和证据生成质量与诊断准确性之间的关系。
我们进行了一项涉及 86 名医学生（平均 26.03 岁，SD 4.71）的实验研究，重点是呼吸困难病例的病史采集。在这项重复测量研究中，参与者使用 SPs 和 VPs 解决了三个病例。在每个案例之后，学生提供诊断并对感知真实性和认知负荷进行评分。提供的诊断根据诊断准确性进行评分；学生提出的问题根据其证据生成质量进行评分。除了常规的零假设检验外，本研究还使用等效性检验来研究无意义效应的缺失。
SPs 的感知真实性（1 尾 t=11.12；P<.001）高于 VPs。诊断准确性与感知真实性之间的相关性很小（r=0.05），并且既不等效（P=.09）也不具有统计学意义（P=.32）。两种评估方法的认知负荷等效（t=2.81；P=.003）。内在认知负荷（1 尾 r=-0.30；P=.003）和外在负荷（1 尾 r=-0.29；P=.003）与诊断准确性的综合评分呈负相关。证据生成质量与 VPs 的诊断准确性呈正相关（1 尾 r=0.38；P<.001）；对于 SPs，这一发现并不成立（1 尾 r=0.05；P=.32）。将两种评估方法相互比较，SPs 的诊断准确性高于 VPs（2 尾 t=2.49；P=.01）。
感知真实性的结果表明，学习者认为 SPs 比 VPs 更真实。由于较高的内在和外在认知负荷对表现不利，因此必须在评估中系统地监测和操纵这两种类型的认知负荷。SPs 的诊断准确性高于 VPs，这可能会对学生使用 VPs 的成绩产生负面影响。我们确定并讨论了这两种评估方法之间出现这种性能差异的可能原因。
