Centre for Emotional Health, Department of Psychology, Macquarie University.
Department of Psychology, University of Pittsburgh.
Multivariate Behav Res. 2021 Mar-Apr;56(2):368-376. doi: 10.1080/00273171.2021.1886897. Epub 2021 Feb 18.
We recently wrote an article comparing the conclusions that followed from two different approaches to quantifying the reliability and replicability of psychopathology symptom networks. Two commentaries on the article have raised five core criticisms, which are addressed in this response with supporting evidence. 1) We did not over-generalize about the replicability of symptom networks, but rather focused on interpreting the contradictory conclusions of the two sets of methods we examined. 2) We closely followed established recommendations when estimating and interpreting the networks. 3) We also closely followed the relevant tutorials, and used examples interpreted by experts in the field, to interpret the bootnet and NetworkComparisonTest results. 4) It is possible for statistical control to increase reliability, but that does not appear to be the case here. 5) Distinguishing between statistically significant versus substantive differences makes it clear that the differences between the networks affect the inferences we would make about symptom-level relationships (i.e., the basis of the purported utility of symptom networks). Ultimately, there is an important point of agreement between our article and the commentaries: All of these applied examples of cross-sectional symptom networks are demonstrating unreliable parameter estimates. While the commentaries propose that the resulting differences between networks are not genuine or meaningful because they are not statistically significant, we propose that the unreplicable inferences about the symptom-level relationships of interest fundamentally undermine the utility of the symptom networks.
我们最近写了一篇文章，比较了两种不同方法得出的关于量化精神病理学症状网络可靠性和可重复性的结论。该文章有两篇评论，提出了五个核心批评意见，我们在本回应中用证据进行了支持。1）我们没有对症状网络的可重复性进行过度概括，而是专注于解释我们所检查的两种方法得出的矛盾结论。2）我们在估计和解释网络时严格遵循既定建议。3）我们还严格遵循相关教程，并使用该领域专家解释的示例来解释 bootnet 和 NetworkComparisonTest 结果。4）统计控制可以提高可靠性，但在这种情况下似乎并非如此。5）区分统计上显著与实质性差异使得清楚的是，网络之间的差异会影响我们对症状层面关系的推断（即症状网络所谓效用的基础）。最终，我们的文章和评论之间有一个重要的共识：所有这些横断面症状网络的应用实例都表明可靠性参数估计不可靠。虽然评论认为网络之间的差异不是真实或有意义的，因为它们在统计学上不显著，但我们认为，对感兴趣的症状层面关系的不可复制推断从根本上破坏了症状网络的效用。
