Department of Experimental Psychology, University of Oxford, Radcliffe Observatory Quarter, Oxford, OX2 6GG, UK.
J Neurol. 2022 Jan;269(1):233-242. doi: 10.1007/s00415-021-10612-8. Epub 2021 May 29.
Multiverse analysis provides an ideal tool for understanding how inherent, yet ultimately arbitrary methodological choices impact the conclusions of individual studies. With this investigation, we aimed to demonstrate the utility of multiverse analysis for evaluating generalisability and identifying potential sources of bias within studies employing neurological populations.
Multiverse analysis was used to evaluate the robustness of the relationship between post-stroke visuospatial neglect and poor long-term recovery outcome within a sample of 1113 (age = 72.5, 45.1% female) stroke survivors. A total of 25,600 t-test comparisons were run across 400 different patient groups defined using various combinations of valid inclusion criteria based on lesion location, stroke type, assessment time, neglect impairment definition, and scoring criteria across 16 standardised outcome measures.
Overall, 33.9% of conducted comparisons yielded significant results. 99.9% of these significant results fell below the null specification curve, indicating a highly robust relationship between neglect and poor recovery outcome. However, the strength of this effect was not constant across all comparison groups. Comparisons which included < 100 participants, pre-selected patients based on lesion type, or failed to account for allocentric neglect impairment were found to yield average effect sizes which differed substantially. Similarly, average effect sizes differed across various outcome measures with the strongest average effect in comparisons involving an activities of daily living measure and the weakest in comparisons employing a depression subscale.
This investigation demonstrates the utility of multiverse analysis techniques for evaluating effect robustness and identifying potential sources of bias within neurological research.
多元宇宙分析为理解内在的、但最终任意的方法选择如何影响个别研究的结论提供了理想的工具。通过本研究，我们旨在展示多元宇宙分析在评估神经人群研究的普遍性和识别潜在偏差来源方面的效用。
多元宇宙分析用于评估 1113 名卒中幸存者（年龄=72.5，45.1%为女性）样本中卒中后视觉空间忽视与长期预后不良之间关系的稳健性。在 400 个不同的患者组中进行了 25600 次 t 检验比较，这些患者组是根据病变位置、卒中类型、评估时间、忽视损伤定义和 16 个标准化结局测量中的评分标准，使用各种有效的纳入标准组合定义的。
总体而言，33.9%的比较产生了显著结果。99.9%的这些显著结果低于零假设曲线，表明忽视与预后不良之间存在高度稳健的关系。然而，这种效应的强度并非在所有比较组中都保持不变。包含<100 名参与者、根据病变类型预先选择患者或未能考虑到以自我为中心的忽视损伤的比较，其平均效应大小差异很大。同样，各种结局测量中的平均效应大小也不同，涉及日常生活活动测量的比较的平均效应最强，而采用抑郁亚量表的比较的平均效应最弱。
本研究展示了多元宇宙分析技术在评估效应稳健性和识别神经科学研究中潜在偏差来源方面的效用。
