Institute of Medical Informatics, University of Lübeck, Germany.
Department of IT, University of the Punjab, Lahore, Pakistan.
Comput Biol Med. 2023 Nov;166:107489. doi: 10.1016/j.compbiomed.2023.107489. Epub 2023 Sep 22.
Flow experience is a specific positive and affective state that occurs when humans are completely absorbed in an activity and forget everything else. This state can lead to high performance, well-being, and productivity at work. Few studies have been conducted to determine the human flow experience using physiological wearable sensor devices. Other studies rely on self-reported data.
In this article, we use physiological data collected from 25 subjects with multimodal sensing devices, in particular the Empatica E4 wristband, the Emotiv Epoc X electroencephalography (EEG) headset, and the Biosignalplux RespiBAN - in arithmetic and reading tasks to automatically discriminate between flow and non-flow states using feature engineering and deep feature learning approaches. The most meaningful wearable device for flow detection is determined by comparing the performances of each device. We also investigate the connection between emotions and flow by testing transfer learning techniques involving an emotion recognition-related task on the source domain.
The EEG sensor modalities yielded the best performances with an accuracy of 64.97%, and a macro Averaged F1 (AF1) score of 64.95%. An accuracy of 73.63% and an AF1 score of 72.70% were obtained after fusing all sensor modalities from all devices. Additionally, our proposed transfer learning approach using emotional arousal classification on the DEAP dataset led to an increase in performances with an accuracy of 75.10% and an AF1 score of 74.92%.
The results of this study suggest that effective discrimination between flow and non-flow states is possible with multimodal sensor data. The success of transfer learning using the DEAP emotion dataset as a source domain indicates that emotions and flow are connected, and emotion recognition can be used as a latent task to enhance the performance of flow recognition.
心流体验是一种特定的积极和情感状态，当人类完全沉浸在某项活动中并忘记其他一切时就会出现这种状态。这种状态可以带来更高的工作绩效、幸福感和生产力。很少有研究使用生理可穿戴传感器设备来确定人类的心流体验。其他研究则依赖于自我报告的数据。
在本文中，我们使用从 25 名被试者收集的多模态传感设备生理数据，特别是使用 Empatica E4 腕带、Emotiv Epoc X 脑电图（EEG）耳机和 Biosignalplux RespiBAN，在算术和阅读任务中，使用特征工程和深度特征学习方法自动区分心流和非心流状态。通过比较每种设备的性能，确定最有意义的心流检测可穿戴设备。我们还通过在源域上进行与情绪识别相关的任务测试迁移学习技术，研究情绪与心流之间的联系。
EEG 传感器模态的性能最佳，准确率为 64.97%，宏平均 F1（AF1）得分为 64.95%。融合所有设备的所有传感器模态后，可获得 73.63%的准确率和 72.70%的 AF1 得分。此外，我们使用 DEAP 数据集上的情绪唤醒分类提出的迁移学习方法可将性能提高到准确率 75.10%和 AF1 得分 74.92%。
本研究结果表明，使用多模态传感器数据可以有效地区分心流和非心流状态。使用 DEAP 情绪数据集作为源域的迁移学习的成功表明，情绪和心流是相关的，情绪识别可以作为一种潜在任务来提高心流识别的性能。
