Zuckerman Institute, Columbia University, New York, United States.
Western University, London, Canada.
Elife. 2023 Aug 23;12:e82566. doi: 10.7554/eLife.82566.
Neuroscience has recently made much progress, expanding the complexity of both neural activity measurements and brain-computational models. However, we lack robust methods for connecting theory and experiment by evaluating our new big models with our new big data. Here, we introduce new inference methods enabling researchers to evaluate and compare models based on the accuracy of their predictions of representational geometries: A good model should accurately predict the distances among the neural population representations (e.g. of a set of stimuli). Our inference methods combine novel 2-factor extensions of crossvalidation (to prevent overfitting to either subjects or conditions from inflating our estimates of model accuracy) and bootstrapping (to enable inferential model comparison with simultaneous generalization to both new subjects and new conditions). We validate the inference methods on data where the ground-truth model is known, by simulating data with deep neural networks and by resampling of calcium-imaging and functional MRI data. Results demonstrate that the methods are valid and conclusions generalize correctly. These data analysis methods are available in an open-source Python toolbox (rsatoolbox.readthedocs.io).
神经科学最近取得了很大的进展，扩展了神经活动测量和大脑计算模型的复杂性。然而，我们缺乏通过用新的大数据评估新的大模型来连接理论和实验的稳健方法。在这里，我们介绍了新的推理方法，使研究人员能够根据代表几何形状的预测准确性来评估和比较模型：一个好的模型应该能够准确地预测神经群体表示（例如一组刺激）之间的距离。我们的推理方法结合了交叉验证的新颖 2 因素扩展（以防止过度拟合主体或条件，从而夸大我们对模型准确性的估计）和引导（以实现具有同时向新主体和新条件推广的推断模型比较）。我们通过用深度神经网络模拟数据以及通过钙成像和功能磁共振成像数据的重采样，在已知真实模型的数据上验证了推理方法。结果表明该方法是有效的，结论是正确的。这些数据分析方法可在一个开源 Python 工具箱（rsatoolbox.readthedocs.io）中获得。
