Teh Hui Yie, Wang Kevin I-Kai, Kempa-Liehr Andreas W
Department of Electrical, Computer and Software Engineering, The University of Auckland, Auckland 1142, New Zealand.
Department of Engineering Science and Biomedical Engineering, The University of Auckland, Auckland 1142, New Zealand.
Sensors (Basel). 2025 Aug 1;25(15):4757. doi: 10.3390/s25154757.
Detecting previously unseen anomalies in sensor data is a challenging problem for artificial intelligence when sensor-specific and deployment-specific characteristics of the time series need to be learned from a short calibration period. From the application point of view, this challenge becomes increasingly important because many applications are gravitating towards utilising low-cost sensors for Internet of Things deployments. While these sensors offer cost-effectiveness and customisation, their data quality does not match that of their high-end counterparts. To improve sensor data quality while addressing the challenges of anomaly detection in Internet of Things applications, we present an anomaly detection framework that learns a normality model of sensor data. The framework models the typical behaviour of individual sensors, which is crucial for the reliable detection of sensor data anomalies, especially when dealing with sensors observing significantly different signal characteristics. Our framework learns sensor-specific normality models from a small set of anomaly-free training data while employing an unsupervised feature engineering approach to select statistically significant features. The selected features are subsequently used to train a Local Outlier Factor anomaly detection model, which adaptively determines the boundary separating normal data from anomalies. The proposed anomaly detection framework is evaluated on three real-world public environmental monitoring datasets with heterogeneous sensor readings. The sensor-specific normality models are learned from extremely short calibration periods (as short as the first 3 days or 10% of the total recorded data) and outperform four other state-of-the-art anomaly detection approaches with respect to F1-score (between 5.4% and 9.3% better) and Matthews correlation coefficient (between 4.0% and 7.6% better).
当需要从较短的校准周期中学习时间序列的传感器特定和部署特定特征时，检测传感器数据中先前未见过的异常对于人工智能来说是一个具有挑战性的问题。从应用的角度来看，这一挑战变得越来越重要，因为许多应用正倾向于在物联网部署中使用低成本传感器。虽然这些传感器具有成本效益和可定制性，但其数据质量与高端传感器不匹配。为了提高传感器数据质量并应对物联网应用中的异常检测挑战，我们提出了一个异常检测框架，该框架可以学习传感器数据的正常模型。该框架对单个传感器的典型行为进行建模，这对于可靠检测传感器数据异常至关重要，尤其是在处理观测到显著不同信号特征的传感器时。我们的框架从一小部分无异常的训练数据中学习传感器特定的正常模型，同时采用无监督特征工程方法来选择具有统计显著性的特征。随后，所选特征用于训练局部离群因子异常检测模型，该模型自适应地确定将正常数据与异常数据分开的边界。我们在三个具有异构传感器读数的真实世界公共环境监测数据集上对提出的异常检测框架进行了评估。传感器特定的正常模型是从极短的校准周期（短至前3天或总记录数据的10%）中学习得到的，在F1分数（提高5.4%至9.3%）和马修斯相关系数（提高4.0%至7.6%）方面优于其他四种先进的异常检测方法。
