Baskin Engineering, UC Santa Cruz, 1156 High Street, Santa Cruz, CA, 95064, USA.
Electrical Engineering and Computer Sciences Department, UC Berkeley, Soda Hall, Berkeley, CA, 94709, USA.
Biol Cybern. 2023 Oct;117(4-5):363-372. doi: 10.1007/s00422-023-00967-8. Epub 2023 Jul 4.
We propose a subject-aware contrastive learning deep fusion neural network framework for effectively classifying subjects' confidence levels in the perception of visual stimuli. The framework, called WaveFusion, is composed of lightweight convolutional neural networks for per-lead time-frequency analysis and an attention network for integrating the lightweight modalities for final prediction. To facilitate the training of WaveFusion, we incorporate a subject-aware contrastive learning approach by taking advantage of the heterogeneity within a multi-subject electroencephalogram dataset to boost representation learning and classification accuracy. The WaveFusion framework demonstrates high accuracy in classifying confidence levels by achieving a classification accuracy of 95.7% while also identifying influential brain regions.
我们提出了一种基于主题感知对比学习的深度融合神经网络框架，用于有效地对被试对视觉刺激感知的置信水平进行分类。该框架被称为 WaveFusion，由用于每个导联的时频分析的轻量级卷积神经网络和用于整合最终预测的轻量级模态的注意力网络组成。为了便于训练 WaveFusion，我们利用多被试脑电图数据集的异质性，采用基于主题感知的对比学习方法，以提高表示学习和分类准确性。WaveFusion 框架通过实现 95.7%的分类准确率，同时识别出有影响力的脑区，证明了在分类置信水平方面的高精度。
