Department of Psychiatry, University of Pittsburgh, Pittsburgh, PA, USA.
Semel Institute for Neuroscience and Human Behavior, University of California Los Angeles, Los Angeles, CA, USA.
Neuropsychopharmacology. 2023 Sep;48(10):1465-1474. doi: 10.1038/s41386-023-01621-4. Epub 2023 Jun 19.
In recent years, a replication crisis in psychiatry has led to a growing focus on the impact of researchers' analytic decisions on the results from studies. Multiverse analyses involve examining results across a wide array of possible analytic decisions (e.g., log-transforming variables, number of covariates, or treatment of outliers) and identifying if study results are robust to researchers' analytic decisions. Studies have begun to use multiverse analysis for well-studied relationships that have some heterogeneity in results/conclusions across studies.We examine the well-studied relationship between peripheral inflammatory markers (PIMs; e.g., white blood cell count (WBC) and C-reactive protein (CRP)) and depression severity in the large NHANES dataset (n = 25,962). Specification curve analyses tested the impact of 9 common analytic decisions (comprising of 58,000+ possible combinations) on the association of PIMs and depression severity. Relationships of PIMs and total depression severity are robust to analytic decisions (based on tests of inference jointly examining effect sizes and p-values). However, moderate/large differences are noted in effect sizes based on analytic decisions and the majority of analyses do not result in significant findings, with the percentage of analyses with statistically significant results being 46.1% for WBC and 43.8% for CRP. For associations of PIMs with specific symptoms of depression, some associations (e.g., sleep, appetite) in males (but not females) were robust to analytic decisions. We discuss how multiverse analyses can be used to guide research and also the need for authors, reviewers, and editors to incorporate multiverse analyses to enhance replicability of research findings.
近年来，精神病学领域的复制危机引发了人们对研究人员分析决策对研究结果影响的日益关注。多元宇宙分析涉及检查广泛的可能分析决策（例如，对变量进行对数转换、增加协变量的数量或处理异常值）的结果，并确定研究结果是否对研究人员的分析决策具有稳健性。一些研究已经开始使用多元宇宙分析来研究那些在研究结果/结论方面存在一定异质性的关系。我们在大型 NHANES 数据集（n=25962）中研究了外周炎症标志物（PIM；例如白细胞计数（WBC）和 C 反应蛋白（CRP））与抑郁严重程度之间的关系。规格曲线分析测试了 9 种常见分析决策（包含 58000 多种可能的组合）对 PIMs 和抑郁严重程度之间关联的影响。基于对效应大小和 p 值进行联合检验的推断测试，PIMs 和总抑郁严重程度之间的关系对分析决策具有稳健性。然而，基于分析决策，效应大小存在中等/较大差异，并且大多数分析没有导致显著结果，其中白细胞计数的分析中有统计学意义的结果百分比为 46.1%，C 反应蛋白为 43.8%。对于 PIMs 与抑郁特定症状的关联，在男性（而不是女性）中，一些关联（例如睡眠、食欲）对分析决策具有稳健性。我们讨论了多元宇宙分析如何用于指导研究，以及作者、评论员和编辑如何需要采用多元宇宙分析来提高研究结果的可重复性。
