Division of Emergency Medicine, University of Washington School of Medicine, Seattle, WA, USA.
Acad Emerg Med. 2011 Oct;18 Suppl 2(Suppl 2):S79-86. doi: 10.1111/j.1553-2712.2011.01182.x.
Research in cognition has yielded considerable understanding of the diagnostic reasoning process and its evolution during clinical training. This study sought to determine whether or not this literature could be used to improve the assessment of trainees' diagnostic skill by manipulating testing conditions that encourage different modes of reasoning.
The authors developed an online, vignette-based instrument with two sets of testing instructions. The "first impression" condition encouraged nonanalytic responses while the "directed search" condition prompted structured analytic responses. Subjects encountered six cases under the first impression condition and then six cases under the directed search condition. Each condition had three straightforward (simple) and three ambiguous (complex) cases. Subjects were stratified by clinical experience: novice (third- and fourth-year medical students), intermediate (postgraduate year [PGY] 1 and 2 residents), and experienced (PGY 3 residents and faculty). Two investigators scored the exams independently. Mean diagnostic accuracies were calculated for each group. Differences in diagnostic accuracy and reliability of the examination as a function of the predictor variables were assessed.
The examination was completed by 115 subjects. Diagnostic accuracy was significantly associated with the independent variables of case complexity, clinical experience, and testing condition. Overall, mean diagnostic accuracy and the extent to which the test consistently discriminated between subjects (i.e., yielded reliable scores) was higher when participants were given directed search instructions than when they were given first impression instructions. In addition, the pattern of reliability was found to depend on experience: simple cases offered the best reliability for discriminating between novices, complex cases offered the best reliability for discriminating between intermediate residents, and neither type of case discriminated well between experienced practitioners.
These results yield concrete guidance regarding test construction for the purpose of diagnostic skill assessment. The instruction strategy and complexity of cases selected should depend on the experience level and breadth of experience of the subjects one is attempting to assess.
认知研究已经对诊断推理过程及其在临床培训中的演变有了相当的了解。本研究旨在确定是否可以通过操纵测试条件来鼓励不同的推理模式，从而提高对学员诊断技能的评估。
作者开发了一种基于情景的在线工具，有两种测试指令集。“第一印象”条件鼓励非分析性反应，而“定向搜索”条件则提示进行结构化分析反应。在第一印象条件下，受试者遇到了六个案例，然后在定向搜索条件下遇到了六个案例。每种条件都有三个简单（简单）和三个模糊（复杂）的案例。受试者按临床经验分层：新手（三年级和四年级医学生）、中级（住院医师 1 年级和 2 年级）和经验丰富（住院医师 3 年级和教员）。两名调查员独立评分。为每个组计算平均诊断准确性。评估了考试作为预测变量的函数的诊断准确性和可靠性的差异。
有 115 名受试者完成了考试。诊断准确性与案例复杂性、临床经验和测试条件等独立变量显著相关。总体而言，当参与者获得定向搜索指令时，平均诊断准确性以及测试在多大程度上一致地区分受试者（即产生可靠分数）高于当他们获得第一印象指令时。此外，还发现可靠性模式取决于经验：简单案例在区分新手方面提供了最佳的可靠性，复杂案例在区分中级住院医师方面提供了最佳的可靠性，而这两种类型的案例在区分经验丰富的医生方面都无法很好地进行区分。
这些结果为诊断技能评估的测试构建提供了具体的指导。指令策略和选择的案例复杂性应取决于要评估的受试者的经验水平和经验广度。
