Hashimoto Hiroaki, Kameda Seiji, Maezawa Hitoshi, Oshino Satoru, Tani Naoki, Khoo Hui Ming, Yanagisawa Takufumi, Yoshimine Toshiki, Kishima Haruhiko, Hirata Masayuki
Department of Neurological Diagnosis and Restoration, Graduate School of Medicine, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan.
Department of Neurosurgery, Otemae Hospital, Chuo-Ku Otemae 1-5-34, Osaka, Osaka 540-0008, Japan.
Int J Neural Syst. 2021 Nov;31(11):2050056. doi: 10.1142/S0129065720500562. Epub 2020 Sep 16.
To realize a brain-machine interface to assist swallowing, neural signal decoding is indispensable. Eight participants with temporal-lobe intracranial electrode implants for epilepsy were asked to swallow during electrocorticogram (ECoG) recording. Raw ECoG signals or certain frequency bands of the ECoG power were converted into images whose vertical axis was electrode number and whose horizontal axis was time in milliseconds, which were used as training data. These data were classified with four labels (Rest, Mouth open, Water injection, and Swallowing). Deep transfer learning was carried out using AlexNet, and power in the high-[Formula: see text] band (75-150[Formula: see text]Hz) was the training set. Accuracy reached 74.01%, sensitivity reached 82.51%, and specificity reached 95.38%. However, using the raw ECoG signals, the accuracy obtained was 76.95%, comparable to that of the high-[Formula: see text] power. We demonstrated that a version of AlexNet pre-trained with visually meaningful images can be used for transfer learning of visually meaningless images made up of ECoG signals. Moreover, we could achieve high decoding accuracy using the raw ECoG signals, allowing us to dispense with the conventional extraction of high-[Formula: see text] power. Thus, the images derived from the raw ECoG signals were equivalent to those derived from the high-[Formula: see text] band for transfer deep learning.
为实现用于辅助吞咽的脑机接口，神经信号解码必不可少。八名因癫痫而植入颞叶颅内电极的参与者在进行皮质脑电图（ECoG）记录时被要求进行吞咽。原始的ECoG信号或ECoG功率的特定频段被转换为图像，其纵轴为电极编号，横轴为以毫秒为单位的时间，这些图像用作训练数据。这些数据被分为四个标签（静息、张嘴、注水和吞咽）。使用AlexNet进行深度迁移学习，以高频带（75 - 150Hz）的功率作为训练集。准确率达到74.01%，灵敏度达到82.51%，特异性达到95.38%。然而，使用原始的ECoG信号时，获得的准确率为76.95%，与高频功率的准确率相当。我们证明，用视觉上有意义的图像预训练的AlexNet版本可用于由ECoG信号组成的视觉上无意义的图像的迁移学习。此外，我们使用原始的ECoG信号能够实现高解码准确率，从而无需传统的高频功率提取。因此，从原始ECoG信号导出的图像对于迁移深度学习而言等同于从高频带导出的图像。
