Midwifery Research and Education Unit, Hannover Medical School, Hannover, 30625, Germany.
Syst Rev. 2024 Oct 17;13(1):261. doi: 10.1186/s13643-024-02680-4.
The standard approach to local inconsistency assessment typically relies on testing the conflict between the direct and indirect evidence in selected treatment comparisons. However, statistical tests for inconsistency have low power and are subject to misinterpreting a p-value above the significance threshold as evidence of consistency.
We propose a simple framework to interpret local inconsistency based on the average Kullback-Leibler divergence (KLD) from approximating the direct with the corresponding indirect estimate and vice versa. Our framework uses directly the mean and standard error (or posterior mean and standard deviation) of the direct and indirect estimates obtained from a local inconsistency method to calculate the average KLD measure for selected comparisons. The average KLD values are compared with a semi-objective threshold to judge the inconsistency as acceptably low or material. We exemplify our novel interpretation approach using three networks with multiple treatments and multi-arm studies.
Almost all selected comparisons in the networks were not associated with statistically significant inconsistency at a significance level of 5%. The proposed interpretation framework indicated 14%, 66%, and 75% of the selected comparisons with an acceptably low inconsistency in the corresponding networks. Overall, information loss was more notable when approximating the posterior density of the indirect estimates with that of the direct estimates, attributed to indirect estimates being more imprecise.
Using the concept of information loss between two distributions alongside a semi-objectively defined threshold helped distinguish target comparisons with acceptably low inconsistency from those with material inconsistency when statistical tests for inconsistency were inconclusive.
局部不一致性评估的标准方法通常依赖于检验选定治疗比较中直接证据和间接证据之间的冲突。然而，不一致性的统计检验功效较低，并且容易将显著性阈值以上的 p 值错误解释为一致性的证据。
我们提出了一种简单的框架，基于直接估计与相应间接估计之间的平均 Kullback-Leibler 散度（KLD）来解释局部不一致性。我们的框架直接使用局部不一致性方法获得的直接和间接估计的均值和标准误差（或后验均值和标准偏差）来计算所选比较的平均 KLD 度量。将平均 KLD 值与半客观阈值进行比较，以判断不一致性是否可接受地低或显著。我们使用三个具有多种治疗和多臂研究的网络来举例说明我们的新解释方法。
在显著性水平为 5%时，网络中几乎所有选定的比较都与统计学上显著的不一致性无关。在所涉及的网络中，分别有 14%、66%和 75%的选定比较被认为具有可接受的低不一致性。总体而言，当用直接估计的后验密度来近似间接估计的后验密度时，信息损失更为明显，这归因于间接估计更不精确。
使用两个分布之间的信息损失概念以及半客观定义的阈值，有助于在不一致性统计检验不确定时，区分具有可接受低不一致性的目标比较和具有显著不一致性的目标比较。
