Department of Engineering Science, University of Oxford, Information Engineering Building, Oxford, OX1 3PJ, United Kingdom.
Department of Molecular and Cellular Biology, School of Engineering and Applied Sciences and Center for Brain Science, Harvard University, 52 Oxford St., Cambridge, MA, 02138, USA.
Sci Rep. 2018 Mar 29;8(1):5397. doi: 10.1038/s41598-018-23618-6.
Machine learning is a field of computer science that builds algorithms that learn. In many cases, machine learning algorithms are used to recreate a human ability like adding a caption to a photo, driving a car, or playing a game. While the human brain has long served as a source of inspiration for machine learning, little effort has been made to directly use data collected from working brains as a guide for machine learning algorithms. Here we demonstrate a new paradigm of "neurally-weighted" machine learning, which takes fMRI measurements of human brain activity from subjects viewing images, and infuses these data into the training process of an object recognition learning algorithm to make it more consistent with the human brain. After training, these neurally-weighted classifiers are able to classify images without requiring any additional neural data. We show that our neural-weighting approach can lead to large performance gains when used with traditional machine vision features, as well as to significant improvements with already high-performing convolutional neural network features. The effectiveness of this approach points to a path forward for a new class of hybrid machine learning algorithms which take both inspiration and direct constraints from neuronal data.
机器学习是计算机科学的一个领域，它构建的算法可以学习。在许多情况下，机器学习算法被用于重现人类的能力，例如给照片添加标题、驾驶汽车或玩游戏。虽然人类大脑长期以来一直是机器学习的灵感来源，但很少有人努力直接使用从工作大脑中收集的数据作为机器学习算法的指导。在这里，我们展示了一种新的“神经加权”机器学习范例，它从观看图像的受试者的 fMRI 测量中获取人类大脑活动，并将这些数据注入到对象识别学习算法的训练过程中，使其更符合人脑。训练后，这些神经加权分类器可以在不需要任何额外神经数据的情况下对图像进行分类。我们表明，当与传统的机器视觉特征一起使用时，我们的神经加权方法可以带来性能的大幅提升，并且已经具有高性能的卷积神经网络特征也可以得到显著改善。这种方法的有效性为一类新的混合机器学习算法指明了前进的道路，这些算法既从神经元数据中获取灵感，又受到直接约束。
