Xu Jinlong, Guo Xinge, Zhang Zixuan, Liu Huajun, Lee Chengkuo
Department of Electrical and Computer Engineering, National University of Singapore, Singapore, 117583, Singapore.
Center for Intelligent Sensors and MEMS (CISM), National University of Singapore, Singapore, 117608, Singapore.
Adv Sci (Weinh). 2025 Feb;12(6):e2407888. doi: 10.1002/advs.202407888. Epub 2024 Dec 19.
To implement digital-twin smart home applications, the mat sensing system based on triboelectric sensors is commonly used for gait information collection from daily activities. Yet traditional mat sensing systems often miss upper body motions and fail to adequately project these into the virtual realm, limiting their specific application scenarios. Herein, triboelectric mat multimodal sensing system is designed, enhanced with a commercial infrared imaging sensor, to capture diverse sensory information for sleep and emotion-relevant activity monitoring without compromising privacy. This system generates pixel-based area ratio mappings across the entire mat array, solely based on the integral operation of triboelectric outputs. Additionally, it utilizes multimodal sensory intelligence and deep-learning analytics to detect different sleeping postures and monitor comprehensive sleep behaviors and emotional states associated with daily activities. These behaviors are projected into the metaverse, enhancing virtual interactions. This multimodal sensing system, cost-effective and non-intrusive, serves as a functional interface for diverse digital-twin smart home applications such as healthcare, sports monitoring, and security.
为了实现数字孪生智能家居应用，基于摩擦电传感器的地垫传感系统通常用于从日常活动中收集步态信息。然而，传统的地垫传感系统往往会遗漏上半身的动作，并且无法将这些动作充分投射到虚拟领域，从而限制了它们的特定应用场景。在此，设计了一种摩擦电地垫多模态传感系统，并通过商用红外成像传感器进行增强，以在不影响隐私的情况下捕获用于睡眠和与情绪相关活动监测的各种感官信息。该系统仅基于摩擦电输出的积分运算，在整个地垫阵列上生成基于像素的面积比映射。此外，它利用多模态感官智能和深度学习分析来检测不同的睡眠姿势，并监测与日常活动相关的综合睡眠行为和情绪状态。这些行为被投射到元宇宙中，增强了虚拟交互。这种经济高效且非侵入性的多模态传感系统，可作为医疗保健、运动监测和安全等各种数字孪生智能家居应用的功能接口。
