Brouwer Anne-Marie, van der Waa Jasper, Stokking Hans
Department of Perceptual and Cognitive Systems, Netherlands Organization for Applied Scientific Research (TNO), Soesterberg, Netherlands.
Department of Media Networking, Netherlands Organization for Applied Scientific Research (TNO), Den Haag, Netherlands.
Front Hum Neurosci. 2018 Oct 16;12:420. doi: 10.3389/fnhum.2018.00420. eCollection 2018.
While numerous studies show that brain signals contain information about an individual's current state that are potentially valuable for smoothing man-machine interfaces, this has not yet lead to the use of brain computer interfaces (BCI) in daily life. One of the main challenges is the common requirement of personal data that is correctly labeled concerning the state of interest in order to train a model, where this trained model is not guaranteed to generalize across time and context. Another challenge is the requirement to wear electrodes on the head. We here propose a BCI that can tackle these issues and may be a promising case for BCI research and application in everyday life. The BCI uses EEG signals to predict head rotation in order to improve images presented in a virtual reality (VR) headset. When presenting a 360° video to a headset, field-of-view approaches only stream the content that is in the current field of view and leave out the rest. When the user rotates the head, other content parts need to be made available soon enough to go unnoticed by the user, which is problematic given the available bandwidth. By predicting head rotation, the content parts adjacent to the currently viewed part could be retrieved in time for display when the rotation actually takes place. We here studied whether head rotations can be predicted on the basis of EEG sensor data and if so, whether application of such predictions could be applied to improve display of streaming images. Eleven participants generated left- and rightward head rotations while head movements were recorded using the headsets motion sensing system and EEG. We trained neural network models to distinguish EEG epochs preceding rightward, leftward, and no rotation. Applying these models to streaming EEG data that was withheld from the training showed that 400 ms before rotation onset, the probability "no rotation" started to decrease and the probabilities of an upcoming right- or leftward rotation started to diverge in the correct direction. In the proposed BCI scenario, users already wear a device on their head allowing for integrated EEG sensors. Moreover, it is possible to acquire accurately labeled training data on the fly, and continuously monitor and improve the model's performance. The BCI can be harnessed if it will improve imagery and therewith enhance immersive experience.
虽然众多研究表明，大脑信号包含有关个人当前状态的信息，这些信息对于优化人机界面可能具有重要价值，但这尚未促使脑机接口（BCI）在日常生活中得到应用。主要挑战之一是，为了训练模型，通常需要正确标记与感兴趣状态相关的个人数据，而这种经过训练的模型并不能保证在不同时间和情境下都能通用。另一个挑战是需要在头部佩戴电极。我们在此提出一种能够解决这些问题的BCI，它可能是BCI在日常生活中进行研究和应用的一个有前景的案例。该BCI使用脑电图（EEG）信号来预测头部转动，以改善虚拟现实（VR）头显中呈现的图像。当向头显播放360°视频时，视场方法仅传输当前视场中的内容，而忽略其余部分。当用户转动头部时，其他内容部分需要尽快可用，以便用户不会注意到，鉴于可用带宽，这是个问题。通过预测头部转动，可以在转动实际发生时及时检索与当前查看部分相邻的内容部分以进行显示。我们在此研究了是否可以基于EEG传感器数据预测头部转动，如果可以，这种预测的应用是否可以用于改善流式图像的显示。11名参与者在使用头显运动传感系统和EEG记录头部运动的同时，进行了向左和向右的头部转动。我们训练神经网络模型以区分向右、向左和无转动之前的EEG时段。将这些模型应用于未用于训练的流式EEG数据表明，在转动开始前400毫秒，“无转动”的概率开始下降，即将到来的向右或向左转动的概率开始在正确方向上发散。在所提出的BCI场景中，用户已经在头部佩戴了允许集成EEG传感器的设备。此外，有可能即时获取准确标记的训练数据，并持续监控和改善模型的性能。如果该BCI能够改善图像并从而增强沉浸式体验，就可以加以利用。
