Department of Computer Science and Technology, Tsinghua University, Beijing, China.
Department of Computer Science and Technology, Harbin Institute of Technology, Herbin, China.
J Biomed Inform. 2020 Jun;106:103427. doi: 10.1016/j.jbi.2020.103427. Epub 2020 Apr 25.
Psychological stress turns out to be increasingly severe among teenagers and has imposed numerous physical and mental issues on them. The earlier the stress is detected, the better it can be effectively managed and alleviated. Smart phones, having taken up an integral part of our daily lives, can act as a way to monitor and collect people's daily behaviors and help people manage stress. In the present study, a Multi-modal Interactive Fusion Method (MIFM) was proposed to detect psychological stress from three types of data (namely, texts, images, sleep and exercise data) harvested using a self-developed mobile app termed as Happort. The method characterized the associations between each two modalities and calculated the contribution of each modality via two attention mechanisms. As revealed from our experimental results, fusing multi-modalities for stress detection exhibits consistently higher performance than using single-modality.
心理压力在青少年中日益严重，给他们带来了许多身心问题。及早发现压力，就能更好地对其进行有效管理和缓解。智能手机已经成为我们日常生活不可或缺的一部分，它可以作为一种监测和收集人们日常行为的方式，帮助人们管理压力。在本研究中，提出了一种多模态交互融合方法（MIFM），用于从使用名为 Happort 的自制移动应用程序采集的三种类型的数据（即文本、图像、睡眠和运动数据）中检测心理压力。该方法刻画了每两种模态之间的关联，并通过两个注意力机制计算了每种模态的贡献。实验结果表明，多模态融合进行压力检测的性能始终优于单模态。
