Wallace H. Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, United States of America.
Department of Neuroscience, Northwestern University, Chicago, IL, United States of America.
J Neural Eng. 2024 Apr 17;21(2):026046. doi: 10.1088/1741-2552/ad3b3a.
Artificial neural networks (ANNs) are state-of-the-art tools for modeling and decoding neural activity, but deploying them in closed-loop experiments with tight timing constraints is challenging due to their limited support in existing real-time frameworks. Researchers need a platform that fully supports high-level languages for running ANNs (e.g. Python and Julia) while maintaining support for languages that are critical for low-latency data acquisition and processing (e.g. C and C++).To address these needs, we introduce the Backend for Realtime Asynchronous Neural Decoding (BRAND). BRAND comprises Linux processes, termed, which communicate with each other in avia streams of data. Its asynchronous design allows for acquisition, control, and analysis to be executed in parallel on streams of data that may operate at different timescales. BRAND uses Redis, an in-memory database, to send data between nodes, which enables fast inter-process communication and supports 54 different programming languages. Thus, developers can easily deploy existing ANN models in BRAND with minimal implementation changes.In our tests, BRAND achieved <600 microsecond latency between processes when sending large quantities of data (1024 channels of 30 kHz neural data in 1 ms chunks). BRAND runs a brain-computer interface with a recurrent neural network (RNN) decoder with less than 8 ms of latency from neural data input to decoder prediction. In a real-world demonstration of the system, participant T11 in the BrainGate2 clinical trial (ClinicalTrials.gov Identifier: NCT00912041) performed a standard cursor control task, in which 30 kHz signal processing, RNN decoding, task control, and graphics were all executed in BRAND. This system also supports real-time inference with complex latent variable models like Latent Factor Analysis via Dynamical Systems.By providing a framework that is fast, modular, and language-agnostic, BRAND lowers the barriers to integrating the latest tools in neuroscience and machine learning into closed-loop experiments.
人工神经网络（ANNs）是建模和解码神经活动的最先进工具，但由于它们在现有实时框架中的支持有限，因此在具有严格时间限制的闭环实验中部署它们具有挑战性。研究人员需要一个完全支持用于运行 ANN 的高级语言（例如 Python 和 Julia）的平台，同时保持对关键低延迟数据采集和处理语言（例如 C 和 C++）的支持。
为了满足这些需求，我们引入了实时异步神经解码后端（BRAND）。BRAND 由 Linux 进程组成，这些进程通过数据流进行通信。它的异步设计允许在不同时间尺度上运行的数据流上并行执行采集、控制和分析。BRAND 使用 Redis（一种内存数据库）在节点之间发送数据，这实现了快速的进程间通信并支持 54 种不同的编程语言。因此，开发人员可以轻松地在 BRAND 中部署现有的 ANN 模型，而无需进行大量的实现更改。
在我们的测试中，当发送大量数据（1 ms 块中有 1024 个 30 kHz 神经数据通道）时，BRAND 中进程之间的延迟小于 600 微秒。BRAND 使用递归神经网络（RNN）解码器运行脑机接口，从神经数据输入到解码器预测的延迟小于 8 ms。在该系统的实际演示中，BrainGate2 临床试验（ClinicalTrials.gov 标识符：NCT00912041）中的参与者 T11 执行了一项标准光标控制任务，其中 30 kHz 信号处理、RNN 解码、任务控制和图形均在 BRAND 中执行。该系统还通过使用 Dynamical Systems 支持复杂潜在变量模型（例如潜在因子分析）的实时推理。
通过提供快速、模块化且与语言无关的框架，BRAND 降低了将最新的神经科学和机器学习工具集成到闭环实验中的门槛。
