Educational Physiology Laboratory, Graduate School of Education, The University of Tokyo, Tokyo, Japan.
Technical University of Munich, Munich, Germany.
JMIR Ment Health. 2024 Oct 18;11:e59512. doi: 10.2196/59512.
The field of mental health technology presently has significant gaps that need addressing, particularly in the domain of daily monitoring and personalized assessments. Current noninvasive devices such as wristbands and smartphones are capable of collecting a wide range of data, which has not yet been fully used for mental health monitoring.
This study aims to introduce a novel dataset for personalized daily mental health monitoring and a new macro-micro framework. This framework is designed to use multimodal and multitask learning strategies for improved personalization and prediction of emotional states in individuals.
Data were collected from 298 individuals using wristbands and smartphones, capturing physiological signals, speech data, and self-annotated emotional states. The proposed framework combines macro-level emotion transformer embeddings with micro-level personalization layers specific to each user. It also introduces a Dynamic Restrained Uncertainty Weighting method to effectively integrate various data types for a balanced representation of emotional states. Several fusion techniques, personalization strategies, and multitask learning approaches were explored.
The proposed framework was evaluated using the concordance correlation coefficient, resulting in a score of 0.503. This result demonstrates the framework's efficacy in predicting emotional states.
The study concludes that the proposed multimodal and multitask learning framework, which leverages transformer-based techniques and dynamic task weighting strategies, is superior for the personalized monitoring of mental health. The study indicates the potential of transforming daily mental health monitoring into a more personalized app, opening up new avenues for technology-based mental health interventions.
心理健康技术领域目前存在重大差距，需要加以解决，特别是在日常监测和个性化评估领域。目前的非侵入性设备，如腕带和智能手机，能够收集广泛的数据，但这些数据尚未被充分用于心理健康监测。
本研究旨在引入一个用于个性化日常心理健康监测的新数据集和一个新的宏微观框架。该框架旨在使用多模态和多任务学习策略来提高个性化和预测个体情绪状态的能力。
使用腕带和智能手机从 298 名个体中收集数据，捕捉生理信号、语音数据和自我标注的情绪状态。所提出的框架将宏观级别的情感转换器嵌入与每个用户特定的微观级别的个性化层相结合。它还引入了一种动态受限不确定性加权方法，以有效地整合各种数据类型，实现情绪状态的平衡表示。探索了几种融合技术、个性化策略和多任务学习方法。
使用一致性相关系数评估所提出的框架，得到了 0.503 的分数。这一结果表明了该框架在预测情绪状态方面的有效性。
研究得出结论，所提出的基于转换器的多模态和多任务学习框架，以及利用动态任务加权策略，非常适合个性化的心理健康监测。该研究表明，将日常心理健康监测转化为更个性化的应用程序具有潜力，为基于技术的心理健康干预开辟了新途径。
