Finke Andrea, Essig Kai, Marchioro Giuseppe, Ritter Helge
Center of Excellence Cognitive Interaction Technology CITEC, Bielefeld University, Bielefeld, Germany.
Neuroinformatics Group, Technical Faculty, Bielefeld University, Bielefeld, Germany.
PLoS One. 2016 Jan 26;11(1):e0146848. doi: 10.1371/journal.pone.0146848. eCollection 2016.
The co-registration of eye tracking and electroencephalography provides a holistic measure of ongoing cognitive processes. Recently, fixation-related potentials have been introduced to quantify the neural activity in such bi-modal recordings. Fixation-related potentials are time-locked to fixation onsets, just like event-related potentials are locked to stimulus onsets. Compared to existing electroencephalography-based brain-machine interfaces that depend on visual stimuli, fixation-related potentials have the advantages that they can be used in free, unconstrained viewing conditions and can also be classified on a single-trial level. Thus, fixation-related potentials have the potential to allow for conceptually different brain-machine interfaces that directly interpret cortical activity related to the visual processing of specific objects. However, existing research has investigated fixation-related potentials only with very restricted and highly unnatural stimuli in simple search tasks while participant's body movements were restricted. We present a study where we relieved many of these restrictions while retaining some control by using a gaze-contingent visual search task. In our study, participants had to find a target object out of 12 complex and everyday objects presented on a screen while the electrical activity of the brain and eye movements were recorded simultaneously. Our results show that our proposed method for the classification of fixation-related potentials can clearly discriminate between fixations on relevant, non-relevant and background areas. Furthermore, we show that our classification approach generalizes not only to different test sets from the same participant, but also across participants. These results promise to open novel avenues for exploiting fixation-related potentials in electroencephalography-based brain-machine interfaces and thus providing a novel means for intuitive human-machine interaction.
眼动追踪与脑电图的同步记录为正在进行的认知过程提供了一种全面的测量方法。最近，与注视相关的电位已被引入，用于量化这种双模态记录中的神经活动。与事件相关电位锁定刺激起始一样，与注视相关的电位在时间上锁定注视起始。与现有的基于脑电图的脑机接口依赖视觉刺激相比，与注视相关的电位具有可在自由、无约束的观看条件下使用且也可在单次试验水平上进行分类的优点。因此，与注视相关的电位有可能实现概念上不同的脑机接口，直接解释与特定物体视觉处理相关的皮层活动。然而，现有研究仅在简单搜索任务中使用非常受限且极不自然的刺激来研究与注视相关的电位，同时参与者的身体动作也受到限制。我们开展了一项研究，通过使用基于注视的视觉搜索任务，在保留一定控制的同时减轻了许多这些限制。在我们的研究中，参与者必须在屏幕上呈现的12个复杂日常物体中找到目标物体，同时同步记录大脑的电活动和眼动。我们的结果表明，我们提出的与注视相关电位的分类方法能够清晰地区分对相关区域、非相关区域和背景区域的注视。此外，我们表明我们的分类方法不仅能推广到同一参与者的不同测试集，还能推广到不同参与者。这些结果有望为在基于脑电图的脑机接口中利用与注视相关的电位开辟新途径，从而为直观的人机交互提供一种新手段。
