Department of Biomedical Engineering, Columbia University, New York, NY 10027, USA.
J Neural Eng. 2011 Jun;8(3):036025. doi: 10.1088/1741-2560/8/3/036025. Epub 2011 May 12.
We describe a closed-loop brain-computer interface that re-ranks an image database by iterating between user generated 'interest' scores and computer vision generated visual similarity measures. The interest scores are based on decoding the electroencephalographic (EEG) correlates of target detection, attentional shifts and self-monitoring processes, which result from the user paying attention to target images interspersed in rapid serial visual presentation (RSVP) sequences. The highest scored images are passed to a semi-supervised computer vision system that reorganizes the image database accordingly, using a graph-based representation that captures visual similarity between images. The system can either query the user for more information, by adaptively resampling the database to create additional RSVP sequences, or it can converge to a 'done' state. The done state includes a final ranking of the image database and also a 'guess' of the user's chosen category of interest. We find that the closed-loop system's re-rankings can substantially expedite database searches for target image categories chosen by the subjects. Furthermore, better reorganizations are achieved than by relying on EEG interest rankings alone, or if the system were simply run in an open loop format without adaptive resampling.
我们描述了一个闭环脑机接口，通过在用户生成的“兴趣”评分和计算机视觉生成的视觉相似性度量之间迭代，对图像数据库进行重新排序。兴趣评分基于解码目标检测、注意力转移和自我监控过程的脑电图 (EEG) 相关物，这些过程是由用户关注快速序列视觉呈现 (RSVP) 序列中穿插的目标图像引起的。得分最高的图像被传递给一个半监督计算机视觉系统，该系统使用基于图的表示法，根据视觉相似性对图像数据库进行相应的重新组织。该系统可以通过自适应地对数据库进行重采样以创建额外的 RSVP 序列来查询用户以获取更多信息，或者可以收敛到“完成”状态。完成状态包括图像数据库的最终排名，以及用户选择的感兴趣类别“猜测”。我们发现，闭环系统的重新排序可以大大加快用户选择的目标图像类别的数据库搜索速度。此外，与仅依赖 EEG 兴趣排名或如果系统仅在没有自适应重采样的开放循环格式下运行相比，可以实现更好的重新组织。
