Flatiron Health, Inc., 233 Spring St, New York, NY, 10013, USA.
Present affiliation: Indigo Ag, Boston, MA, 02129, USA.
BMC Med Res Methodol. 2021 Oct 25;21(1):221. doi: 10.1186/s12874-021-01432-5.
Statistical inference based on small datasets, commonly found in precision oncology, is subject to low power and high uncertainty. In these settings, drawing strong conclusions about future research utility is difficult when using standard inferential measures. It is therefore important to better quantify the uncertainty associated with both significant and non-significant results based on small sample sizes.
We developed a new method, Bayesian Additional Evidence (BAE), that determines (1) how much additional supportive evidence is needed for a non-significant result to reach Bayesian posterior credibility, or (2) how much additional opposing evidence is needed to render a significant result non-credible. Although based in Bayesian analysis, a prior distribution is not needed; instead, the tipping point output is compared to reasonable effect ranges to draw conclusions. We demonstrate our approach in a comparative effectiveness analysis comparing two treatments in a real world biomarker-defined cohort, and provide guidelines for how to apply BAE in practice.
Our initial comparative effectiveness analysis results in a hazard ratio of 0.31 with 95% confidence interval (0.09, 1.1). Applying BAE to this result yields a tipping point of 0.54; thus, an observed hazard ratio of 0.54 or smaller in a replication study would result in posterior credibility for the treatment association. Given that effect sizes in this range are not extreme, and that supportive evidence exists from a similar published study, we conclude that this problem is worthy of further research.
Our proposed method provides a useful framework for interpreting analytic results from small datasets. This can assist researchers in deciding how to interpret and continue their investigations based on an initial analysis that has high uncertainty. Although we illustrated its use in estimating parameters based on time-to-event outcomes, BAE easily applies to any normally-distributed estimator, such as those used for analyzing binary or continuous outcomes.
基于小型数据集的统计推断在精准肿瘤学中很常见，但存在效能低和不确定性高的问题。在这些情况下，使用标准推断方法很难对未来研究的实用性得出强有力的结论。因此，基于小样本量，更好地量化显著和非显著结果相关的不确定性非常重要。
我们开发了一种新方法，贝叶斯附加证据（BAE），用于确定（1）非显著结果达到贝叶斯后验可信度需要多少额外的支持性证据，或（2）需要多少额外的反对性证据才能使显著结果变得不可信。虽然基于贝叶斯分析，但不需要先验分布；相反，将临界点输出与合理的效应范围进行比较以得出结论。我们在一个真实世界生物标志物定义队列中比较两种治疗方法的比较有效性分析中展示了我们的方法，并提供了在实践中应用 BAE 的指南。
我们的初始比较有效性分析得出风险比为 0.31，95%置信区间为（0.09，1.1）。将 BAE 应用于该结果得出临界点为 0.54；因此，在复制研究中观察到的风险比为 0.54 或更小，则治疗相关性将具有后验可信度。鉴于该范围内的效应大小并不极端，并且存在来自类似已发表研究的支持性证据，我们得出结论，该问题值得进一步研究。
我们提出的方法为解释小型数据集的分析结果提供了有用的框架。这可以帮助研究人员根据具有高不确定性的初始分析来决定如何解释和继续他们的研究。虽然我们说明了它在估计基于时间事件结果的参数中的用途，但 BAE 很容易应用于任何正态分布的估计量，例如用于分析二项或连续结果的估计量。
