College of Computer Science and Technology, Jilin University, Changchun 130012, China.
School of Artificial Intelligence, Jilin University, Changchun 130012, China.
Sensors (Basel). 2023 Jan 17;23(3):1083. doi: 10.3390/s23031083.
The ubiquity of smartphones equipped with multiple sensors has provided the possibility of automatically recognizing of human activity, which can benefit intelligent applications such as smart homes, health monitoring, and aging care. However, there are two major barriers to deploying an activity recognition model in real-world scenarios. Firstly, deep learning models for activity recognition use a large amount of sensor data, which are privacy-sensitive and hence cannot be shared or uploaded to a centralized server. Secondly, divergence in the distribution of sensory data exists among multiple individuals due to their diverse behavioral patterns and lifestyles, which contributes to difficulty in recognizing activity for large-scale users or 'cold-starts' for new users. To address these problems, we propose DivAR, a diversity-aware activity recognition framework based on a federated Meta-Learning architecture, which can extract general sensory features shared among individuals by a centralized embedding network and individual-specific features by attention module in each decentralized network. Specifically, we first classify individuals into multiple clusters according to their behavioral patterns and social factors. We then apply meta-learning in the architecture of federated learning, where a centralized meta-model learns common feature representations that can be transferred across all clusters of individuals, and multiple decentralized cluster-specific models are utilized to learn cluster-specific features. For each cluster-specific model, a CNN-based attention module learns cluster-specific features from the global model. In this way, by training with sensory data locally, privacy-sensitive information existing in sensory data can be preserved. To evaluate the model, we conduct two data collection experiments by collecting sensor readings from naturally used smartphones annotated with activity information in the real-life environment and constructing two multi-individual heterogeneous datasets. In addition, social characteristics including personality, mental health state, and behavior patterns are surveyed using questionnaires. Finally, extensive empirical results demonstrate that the proposed diversity-aware activity recognition model has a relatively better generalization ability and achieves competitive performance on multi-individual activity recognition tasks.
智能手机的普及配备了多种传感器，为自动识别人类活动提供了可能性，这可以使智能家居、健康监测和老龄化护理等智能应用受益。然而，在实际场景中部署活动识别模型有两个主要障碍。首先，用于活动识别的深度学习模型使用大量传感器数据，这些数据是隐私敏感的，因此不能共享或上传到集中式服务器。其次，由于个体的行为模式和生活方式不同，多个个体的传感器数据分布存在差异，这导致难以识别大规模用户的活动或新用户的“冷启动”。为了解决这些问题，我们提出了 DivAR，这是一种基于联邦元学习架构的多样性感知活动识别框架，它可以通过集中嵌入网络提取个体之间共享的通用传感器特征，并通过每个分散网络中的注意力模块提取个体特定的特征。具体来说，我们首先根据个体的行为模式和社会因素将个体分为多个簇。然后，我们在联邦学习的架构中应用元学习，其中集中元模型学习可以在所有个体簇中转移的通用特征表示，并且使用多个分散的簇特定模型来学习簇特定的特征。对于每个簇特定的模型，基于 CNN 的注意力模块从全局模型中学习簇特定的特征。这样，通过在本地使用传感器数据进行训练，可以保留传感器数据中存在的隐私敏感信息。为了评估模型，我们通过从真实环境中使用智能手机收集带有活动信息的传感器读数并构建两个多个体异构数据集来进行两个数据收集实验。此外，还使用问卷调查了包括个性、心理健康状况和行为模式在内的社会特征。最后，广泛的实证结果表明，所提出的多样性感知活动识别模型具有相对较好的泛化能力，并在多个体活动识别任务中达到了有竞争力的性能。
