Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA.
Biomed Eng Online. 2013 May 16;12:43. doi: 10.1186/1475-925X-12-43.
A large number of incommensurable metrics are currently used to report the performance of brain-computer interfaces (BCI) used for augmentative and alterative communication (AAC). The lack of standard metrics precludes the comparison of different BCI-based AAC systems, hindering rapid growth and development of this technology. This paper presents a review of the metrics that have been used to report performance of BCIs used for AAC from January 2005 to January 2012. We distinguish between Level 1 metrics used to report performance at the output of the BCI Control Module, which translates brain signals into logical control output, and Level 2 metrics at the Selection Enhancement Module, which translates logical control to semantic control. We recommend that: (1) the commensurate metrics Mutual Information or Information Transfer Rate (ITR) be used to report Level 1 BCI performance, as these metrics represent information throughput, which is of interest in BCIs for AAC; 2) the BCI-Utility metric be used to report Level 2 BCI performance, as it is capable of handling all current methods of improving BCI performance; (3) these metrics should be supplemented by information specific to each unique BCI configuration; and (4) studies involving Selection Enhancement Modules should report performance at both Level 1 and Level 2 in the BCI system. Following these recommendations will enable efficient comparison between both BCI Control and Selection Enhancement Modules, accelerating research and development of BCI-based AAC systems.
目前,大量不可公度的指标被用于报告增强和替代交流(AAC)用脑-机接口(BCI)的性能。缺乏标准的指标使得不同的基于 BCI 的 AAC 系统之间的比较变得不可能,阻碍了这项技术的快速发展。本文回顾了 2005 年 1 月至 2012 年 1 月期间用于报告 AAC 用 BCI 性能的指标。我们区分了用于报告 BCI 控制模块输出性能的一级指标,该模块将脑信号转换为逻辑控制输出,以及用于报告选择增强模块性能的二级指标,该模块将逻辑控制转换为语义控制。我们建议:(1)使用互信息或信息传输率(ITR)等相称的指标来报告一级 BCI 性能,因为这些指标代表了信息吞吐量,这在 AAC 用 BCI 中是很重要的;(2)使用 BCI-效用指标来报告二级 BCI 性能,因为它能够处理所有提高 BCI 性能的当前方法;(3)这些指标应该辅以每个独特的 BCI 配置的具体信息;(4)涉及选择增强模块的研究应该在 BCI 系统的一级和二级报告性能。遵循这些建议将能够在 BCI 控制和选择增强模块之间进行有效的比较,从而加速基于 BCI 的 AAC 系统的研究和开发。