Houston Veterans Affairs Health Services Research and Development Center of Excellence and the Section of Health Services Research, Houston, Texas2Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas3Baylor College of Medicine, Houston, Texas.
JAMA Intern Med. 2013 Nov 25;173(21):1952-8. doi: 10.1001/jamainternmed.2013.10081.
Little is known about the relationship between physicians' diagnostic accuracy and their confidence in that accuracy.
To evaluate how physicians' diagnostic calibration, defined as the relationship between diagnostic accuracy and confidence in that accuracy, changes with evolution of the diagnostic process and with increasing diagnostic difficulty of clinical case vignettes.
DESIGN, SETTING, AND PARTICIPANTS: We recruited general internists from an online physician community and asked them to diagnose 4 previously validated case vignettes of variable difficulty (2 easier; 2 more difficult). Cases were presented in a web-based format and divided into 4 sequential phases simulating diagnosis evolution: history, physical examination, general diagnostic testing data, and definitive diagnostic testing. After each phase, physicians recorded 1 to 3 differential diagnoses and corresponding judgments of confidence. Before being presented with definitive diagnostic data, physicians were asked to identify additional resources they would require to diagnose each case (ie, additional tests, second opinions, curbside consultations, referrals, and reference materials).
Diagnostic accuracy (scored as 0 or 1), confidence in diagnostic accuracy (on a scale of 0-10), diagnostic calibration, and whether additional resources were requested (no or yes).
A total of 118 physicians with broad geographical representation within the United States correctly diagnosed 55.3% of easier and 5.8% of more difficult cases (P < .001). Despite a large difference in diagnostic accuracy between easier and more difficult cases, the difference in confidence was relatively small (7.2 vs 6.4 out of 10, for easier and more difficult cases, respectively) (P < .001) and likely clinically insignificant. Overall, diagnostic calibration was worse for more difficult cases (P < .001) and characterized by overconfidence in accuracy. Higher confidence was related to decreased requests for additional diagnostic tests (P = .01); higher case difficulty was related to more requests for additional reference materials (P = .01).
Our study suggests that physicians' level of confidence may be relatively insensitive to both diagnostic accuracy and case difficulty. This mismatch might prevent physicians from reexamining difficult cases where their diagnosis may be incorrect.
医生的诊断准确性及其对准确性的信心之间的关系知之甚少。
评估医生的诊断校准情况，即诊断准确性与其准确性信心之间的关系，如何随着诊断过程的发展以及临床案例简述的诊断难度的增加而变化。
设计、设置和参与者：我们从在线医生社区招募了普通内科医生，并要求他们诊断 4 个先前经过验证的、难度不同的案例简述（2 个较容易；2 个更难）。案例以基于网络的格式呈现，并分为 4 个连续阶段，模拟诊断的发展：病史、体检、一般诊断测试数据和明确的诊断测试。在每个阶段之后，医生记录 1 到 3 个鉴别诊断和相应的置信度判断。在呈现明确的诊断数据之前，医生被要求确定诊断每个案例所需的其他资源（即，额外的测试、第二意见、床边咨询、转诊和参考资料）。
诊断准确性（得分为 0 或 1）、对诊断准确性的信心（0-10 分）、诊断校准以及是否请求其他资源（否或是）。
来自美国各地的具有广泛地域代表性的 118 名医生正确诊断了 55.3%的较容易案例和 5.8%的更难案例（P<0.001）。尽管较容易和更难案例之间的诊断准确性存在很大差异，但信心差异相对较小（分别为 7.2 和 6.4 分，对于较容易和更难的案例）（P<0.001），且可能在临床上意义不大。总体而言，更难的案例的诊断校准更差（P<0.001），并表现出准确性的过度自信。更高的信心与减少对额外诊断测试的请求有关（P=0.01）；更高的案例难度与更多请求额外参考资料有关（P=0.01）。
我们的研究表明，医生的信心水平可能相对不敏感于诊断准确性和案例难度。这种不匹配可能会阻止医生重新检查他们的诊断可能不正确的困难案例。
