Chen Xiaogang, Li Kun
Institute of Biomedical Engineering, Chinese Academy of Medical Sciences and Peking Union Medical College, Tianjin 300192, P.R.China.
Medical Equipment Management Division of 3201 Hospital, Hanzhong, Shaanxi 723000, P.R.China.
Sheng Wu Yi Xue Gong Cheng Xue Za Zhi. 2021 Jun 25;38(3):483-491. doi: 10.7507/1001-5515.202011039.
Brain-computer interface (BCI) has great potential to replace lost upper limb function. Thus, there has been great interest in the development of BCI-controlled robotic arm. However, few studies have attempted to use noninvasive electroencephalography (EEG)-based BCI to achieve high-level control of a robotic arm. In this paper, a high-level control architecture combining augmented reality (AR) BCI and computer vision was designed to control a robotic arm for performing a pick and place task. A steady-state visual evoked potential (SSVEP)-based BCI paradigm was adopted to realize the BCI system. Microsoft's HoloLens was used to build an AR environment and served as the visual stimulator for eliciting SSVEPs. The proposed AR-BCI was used to select the objects that need to be operated by the robotic arm. The computer vision was responsible for providing the location, color and shape information of the objects. According to the outputs of the AR-BCI and computer vision, the robotic arm could autonomously pick the object and place it to specific location. Online results of 11 healthy subjects showed that the average classification accuracy of the proposed system was 91.41%. These results verified the feasibility of combing AR, BCI and computer vision to control a robotic arm, and are expected to provide new ideas for innovative robotic arm control approaches.
脑机接口（BCI）在替代丧失的上肢功能方面具有巨大潜力。因此，人们对开发由BCI控制的机器人手臂有着浓厚兴趣。然而，很少有研究尝试使用基于非侵入性脑电图（EEG）的BCI来实现对机器人手臂的高级控制。在本文中，设计了一种结合增强现实（AR）BCI和计算机视觉的高级控制架构，以控制机器人手臂执行抓取和放置任务。采用基于稳态视觉诱发电位（SSVEP）的BCI范式来实现BCI系统。微软的HoloLens用于构建AR环境，并作为诱发SSVEP的视觉刺激器。所提出的AR-BCI用于选择需要由机器人手臂操作的物体。计算机视觉负责提供物体的位置、颜色和形状信息。根据AR-BCI和计算机视觉的输出，机器人手臂可以自主抓取物体并将其放置到特定位置。11名健康受试者的在线结果表明，所提出系统的平均分类准确率为91.41%。这些结果验证了结合AR、BCI和计算机视觉来控制机器人手臂的可行性，并有望为创新的机器人手臂控制方法提供新思路。
