Ye Joel, Rizzoglio Fabio, Smoulder Adam, Mao Hongwei, Ma Xuan, Marino Patrick, Chowdhury Raeed, Moore Dalton, Blumenthal Gary, Hockeimer William, Kunigk Nicolas G, Mayo J Patrick, Batista Aaron, Chase Steven, Rouse Adam, Boninger Michael L, Greenspon Charles, Schwartz Andrew B, Hatsopoulos Nicholas G, Miller Lee E, Bouchard Kristofer E, Collinger Jennifer L, Wehbe Leila, Gaunt Robert
Carnegie Mellon University.
Northwestern University.
bioRxiv. 2025 Feb 6:2025.02.02.634313. doi: 10.1101/2025.02.02.634313.
Mapping the relationship between neural activity and motor behavior is a central aim of sensorimotor neuroscience and neurotechnology. While most progress to this end has relied on restricting complexity, the advent of foundation models instead proposes integrating a breadth of data as an alternate avenue for broadly advancing downstream modeling. We quantify this premise for motor decoding from intracortical microelectrode data, pretraining an autoregressive Transformer on 2000 hours of neural population spiking activity paired with diverse motor covariates from over 30 monkeys and humans. The resulting model is broadly useful, benefiting decoding on 8 downstream decoding tasks and generalizing to a variety of neural distribution shifts. However, we also highlight that scaling autoregressive Transformers seems unlikely to resolve limitations stemming from sensor variability and output stereotypy in neural datasets. Code: https://github.com/joel99/ndt3.
绘制神经活动与运动行为之间的关系是感觉运动神经科学和神经技术的核心目标。虽然迄今为止的大多数进展都依赖于限制复杂性，但基础模型的出现提出整合广泛的数据，作为广泛推进下游建模的另一条途径。我们对从皮层内微电极数据进行运动解码的这一前提进行了量化，在2000小时的神经群体尖峰活动与来自30多只猴子和人类的各种运动协变量配对的数据上对自回归Transformer进行预训练。所得模型具有广泛的实用性，有助于在8个下游解码任务上进行解码，并能推广到各种神经分布变化。然而，我们也强调，扩展自回归Transformer似乎不太可能解决神经数据集中传感器变异性和输出刻板性所带来的限制。代码：https://github.com/joel99/ndt3
