Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing 102617, China.
Beijing Academy of Safety Engineering and Technology, Beijing 102617, China.
Sensors (Basel). 2023 May 14;23(10):4750. doi: 10.3390/s23104750.
Human activity recognition (HAR) is becoming increasingly important, especially with the growing number of elderly people living at home. However, most sensors, such as cameras, do not perform well in low-light environments. To address this issue, we designed a HAR system that combines a camera and a millimeter wave radar, taking advantage of each sensor and a fusion algorithm to distinguish between confusing human activities and to improve accuracy in low-light settings. To extract the spatial and temporal features contained in the multisensor fusion data, we designed an improved CNN-LSTM model. In addition, three data fusion algorithms were studied and investigated. Compared to camera data in low-light environments, the fusion data significantly improved the HAR accuracy by at least 26.68%, 19.87%, and 21.92% under the data level fusion algorithm, feature level fusion algorithm, and decision level fusion algorithm, respectively. Moreover, the data level fusion algorithm also resulted in a reduction of the best misclassification rate to 2%~6%. These findings suggest that the proposed system has the potential to enhance the accuracy of HAR in low-light environments and to decrease human activity misclassification rates.
人体活动识别（HAR）变得越来越重要，特别是随着越来越多的老年人在家中生活。然而，大多数传感器，如摄像机，在低光照环境下表现不佳。为了解决这个问题，我们设计了一个结合摄像机和毫米波雷达的 HAR 系统，利用每个传感器和融合算法来区分混淆的人体活动，并提高低光照环境下的准确性。为了提取多传感器融合数据中包含的时空特征，我们设计了一个改进的 CNN-LSTM 模型。此外，研究并调查了三种数据融合算法。与低光照环境下的摄像机数据相比，融合数据在数据级融合算法、特征级融合算法和决策级融合算法下，分别将 HAR 准确性至少提高了 26.68%、19.87%和 21.92%。此外，数据级融合算法还将最佳误分类率降低到 2%~6%。这些发现表明，所提出的系统有可能提高低光照环境下 HAR 的准确性，并降低人体活动误分类率。
