Banga Kush, Benson Julius, Bhagat Jai, Biderman Dan, Birman Daniel, Bonacchi Niccolò, Bruijns Sebastian A, Buchanan Kelly, Campbell Robert A A, Carandini Matteo, Chapuis Gaelle A, Churchland Anne K, Davatolhagh M Felicia, Lee Hyun Dong, Faulkner Mayo, Gerçek Berk, Hu Fei, Huntenburg Julia, Hurwitz Cole Lincoln, Khanal Anup, Krasniak Christopher, Lau Petrina, Langfield Christopher, Mackenzie Nancy, Meijer Guido T, Miska Nathaniel J, Mohammadi Zeinab, Noel Jean-Paul, Paninski Liam, Pan-Vazquez Alejandro, Rossant Cyrille, Roth Noam, Schartner Michael, Socha Karolina Z, Steinmetz Nicholas A, Svoboda Karel, Taheri Marsa, Urai Anne E, Wang Shuqi, Wells Miles, West Steven J, Whiteway Matthew R, Winter Olivier, Witten Ilana B, Zhang Yizi
University College London, London, United Kingdom.
New York University, New York, United States.
Elife. 2025 May 12;13:RP100840. doi: 10.7554/eLife.100840.
Understanding brain function relies on the collective work of many labs generating reproducible results. However, reproducibility has not been systematically assessed within the context of electrophysiological recordings during cognitive behaviors. To address this, we formed a multi-lab collaboration using a shared, open-source behavioral task and experimental apparatus. Experimenters in 10 laboratories repeatedly targeted Neuropixels probes to the same location (spanning secondary visual areas, hippocampus, and thalamus) in mice making decisions; this generated a total of 121 experimental replicates, a unique dataset for evaluating reproducibility of electrophysiology experiments. Despite standardizing both behavioral and electrophysiological procedures, some experimental outcomes were highly variable. A closer analysis uncovered that variability in electrode targeting hindered reproducibility, as did the limited statistical power of some routinely used electrophysiological analyses, such as single-neuron tests of modulation by individual task parameters. Reproducibility was enhanced by histological and electrophysiological quality-control criteria. Our observations suggest that data from systems neuroscience is vulnerable to a lack of reproducibility, but that across-lab standardization, including metrics we propose, can serve to mitigate this.
对脑功能的理解依赖于众多实验室的共同努力，这些实验室要能产生可重复的结果。然而，在认知行为的电生理记录背景下，可重复性尚未得到系统评估。为解决这一问题，我们通过使用共享的开源行为任务和实验设备，组建了一个多实验室合作项目。来自10个实验室的实验人员在小鼠做决策时，将Neuropixels探针反复定位到相同位置（跨越视觉次级区域、海马体和丘脑）；这总共产生了121个实验复制品，这是一个用于评估电生理实验可重复性的独特数据集。尽管对行为和电生理程序都进行了标准化，但一些实验结果仍具有高度变异性。进一步分析发现，电极定位的变异性阻碍了可重复性，一些常用的电生理分析（如单个任务参数调制的单神经元测试）的统计功效有限也起到了同样的作用。组织学和电生理质量控制标准提高了可重复性。我们的观察结果表明，系统神经科学的数据容易出现缺乏可重复性的问题，但跨实验室的标准化，包括我们提出的指标，可以起到缓解这一问题的作用。
