Department of Neurobiology, David Geffen School of Medicine at UCLA, University of California Los Angeles, Los Angeles, California, USA.
Medical Imaging Informatics, Department of Radiological Sciences, University of California Los Angeles, Los Angeles, California, USA.
Eur J Neurosci. 2024 Nov;60(10):6391-6394. doi: 10.1111/ejn.16561. Epub 2024 Oct 15.
The reproducibility crisis highlights several unresolved issues in science, including the need to develop measures that gauge both the consistency and convergence of data sets. While existing meta-analytic methods quantify the consistency of evidence, they do not quantify its convergence: the extent to which different types of empirical methods have provided evidence to support a hypothesis. To address this gap in meta-analysis, we and colleagues developed a summary metric-the cumulative evidence index (CEI)-which uses Bayesian statistics to quantify the degree of both consistency and convergence of evidence regarding causal hypotheses between two phenomena. Here, we outline the CEI's underlying model, which quantifies the extent to which studies of four types-positive intervention, negative intervention, positive non-intervention and negative non-intervention-lend credence to any of three types of causal relations: excitatory, inhibitory or no-connection. Along with p-values and other measures, the CEI can provide a more holistic perspective on a set of evidence by quantitatively expressing epistemic principles that scientists regularly employ qualitatively. The CEI can thus address the reproducibility crisis by formally demonstrating how convergent evidence across multiple study types can yield progress toward scientific consensus, even when an individual type of study fails to yield reproducible results.
可重复性危机凸显了科学中几个未解决的问题，包括需要开发衡量数据集一致性和收敛性的度量标准。虽然现有的元分析方法可以量化证据的一致性，但它们不能量化其收敛性：即不同类型的经验方法在多大程度上提供了证据来支持假设。为了解决元分析中的这一差距，我们和同事开发了一种综合指标——累积证据指数（CEI）——它使用贝叶斯统计来量化关于两个现象之间因果假设的证据的一致性和收敛性程度。在这里，我们概述了 CEI 的基础模型，该模型量化了四种类型的研究——阳性干预、阴性干预、阳性非干预和阴性非干预——在多大程度上支持三种因果关系类型：兴奋、抑制或无连接。与 p 值和其他度量标准一起，CEI 可以通过定量表达科学家经常定性使用的认知原则，为一组证据提供更全面的视角。因此，CEI 可以通过正式证明多种研究类型的收敛性证据如何能够朝着科学共识取得进展，即使个别类型的研究无法产生可重复的结果。
