Division of Ergonomics in the Department of Psychology and Ergonomics (IPA), Technische Universität Berlin, Straße des 17. Juni 135, 10623, Berlin, Germany.
Simply Rational GmbH, Berlin, Germany.
BMC Med Educ. 2023 Feb 7;23(1):75. doi: 10.1186/s12909-023-04025-6.
Many physicians do not know how to accurately interpret test results using Bayes' rule. As a remedy, two kinds of interventions have been shown effective: boosting insight and boosting agency with natural frequencies. To boost insight, test statistics are provided in natural frequencies (rather than conditional probabilities), without instructions on how to use them. To boost agency, a training is provided on how to translate probabilities into natural frequencies and apply them in Bayes' rule. What has not been shown is whether boosting agency is sufficient or if representing test statistics in natural frequencies may additionally boost insight to maximize accurate test interpretation.
We used a pre/posttest design to assess test interpretation accuracy of 577 medical students before and after a training on two Bayesian reasoning tasks, one providing conditional probabilities, the other natural frequencies. The pretest assessed baseline abilities versus the effect of natural frequencies to boost insight. After participants received a training on how to translate conditional probabilities into natural frequencies and how to apply them in Bayes' rule, test interpretation skills were assessed using the same tasks again, comparing the effects of training-induced agency with versus without additionally boosting insight (i.e., test statistics in natural frequencies versus conditional probabilities).
Compared to the test question formatted in conditional probabilities (34% correct answers), natural frequencies facilitated Bayesian reasoning without training (68%), that is, they increased insight. The training on how to use natural frequencies improved performance for tasks formatted in conditional probabilities (64%). Performance was maximal after training and with test statistics formatted in natural frequencies, that is, with a combination of boosting insight and agency (89%).
Natural frequencies should be used to boost insight and agency to maximize effective use of teaching resources. Thus, mandating that test statistics are provided in natural frequencies and adopting short trainings on how to translate conditional probabilities into natural frequencies and how to apply them in Bayes' rule will help to maximize accurate test interpretation.
The study was a registered with the German Clinical Trial Registry ( DRKS00008723 ; 06/03/2015).
许多医生不知道如何使用贝叶斯法则准确解读检验结果。为此，已经证明两种干预措施是有效的：提高洞察力和利用自然频率提高决策能力。为了提高洞察力，是以自然频率（而不是条件概率）提供检验统计数据，而不提供如何使用它们的说明。为了提高决策能力，提供了有关如何将概率转换为自然频率并在贝叶斯规则中应用它们的培训。尚未证明的是，提高决策能力是否足够，或者以自然频率表示检验统计数据是否可以进一步提高洞察力，从而最大限度地提高准确的检验解读。
我们使用前后测试设计，在两项贝叶斯推理任务的培训前后，评估了 577 名医学生的检验解读准确性，其中一项提供条件概率，另一项提供自然频率。在参与者接受了关于如何将条件概率转换为自然频率以及如何在贝叶斯规则中应用它们的培训之后，再次使用相同的任务评估测试解释技能，比较了培训引起的决策能力的影响，同时有无进一步提高洞察力（即，以自然频率表示的检验统计数据与条件概率）。
与以条件概率格式（34%的正确答案）的测试问题相比，没有培训的情况下，自然频率促进了贝叶斯推理（68%），即提高了洞察力。关于如何使用自然频率的培训提高了以条件概率格式的任务表现（64%）。培训后并以自然频率格式呈现检验统计数据时，表现达到最大值，即提高洞察力和决策能力的组合（89%）。
应该使用自然频率来提高洞察力和决策能力，以最大限度地有效利用教学资源。因此，要求以自然频率提供检验统计数据，并采用有关如何将条件概率转换为自然频率以及如何在贝叶斯规则中应用它们的简短培训，将有助于最大限度地提高准确的检验解读。
该研究已在德国临床试验注册处（DRKS00008723；2015 年 3 月 6 日）注册。
