• 文献检索
  • 文档翻译
  • 深度研究
  • 学术资讯
  • Suppr Zotero 插件Zotero 插件
  • 邀请有礼
  • 套餐&价格
  • 历史记录
应用&插件
Suppr Zotero 插件Zotero 插件浏览器插件Mac 客户端Windows 客户端微信小程序
定价
高级版会员购买积分包购买API积分包
服务
文献检索文档翻译深度研究API 文档MCP 服务
关于我们
关于 Suppr公司介绍联系我们用户协议隐私条款
关注我们

Suppr 超能文献

核心技术专利:CN118964589B侵权必究
粤ICP备2023148730 号-1Suppr @ 2026

文献检索

告别复杂PubMed语法,用中文像聊天一样搜索,搜遍4000万医学文献。AI智能推荐,让科研检索更轻松。

立即免费搜索

文件翻译

保留排版,准确专业,支持PDF/Word/PPT等文件格式,支持 12+语言互译。

免费翻译文档

深度研究

AI帮你快速写综述,25分钟生成高质量综述,智能提取关键信息,辅助科研写作。

立即免费体验

NFAD:使用归一化流修复异常检测

NFAD: fixing anomaly detection using normalizing flows.

作者信息

Ryzhikov Artem, Borisyak Maxim, Ustyuzhanin Andrey, Derkach Denis

机构信息

Laboratory of Methods for Big Data Analysis, HSE University, Moscow, Russia.

出版信息

PeerJ Comput Sci. 2021 Nov 18;7:e757. doi: 10.7717/peerj-cs.757. eCollection 2021.

DOI:10.7717/peerj-cs.757
PMID:34901422
原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC8627226/
Abstract

Anomaly detection is a challenging task that frequently arises in practically all areas of industry and science, from fraud detection and data quality monitoring to finding rare cases of diseases and searching for new physics. Most of the conventional approaches to anomaly detection, such as one-class SVM and Robust Auto-Encoder, are one-class classification methods, , focus on separating normal data from the rest of the space. Such methods are based on the assumption of separability of normal and anomalous classes, and subsequently do not take into account any available samples of anomalies. Nonetheless, in practical settings, some anomalous samples are often available; however, usually in amounts far lower than required for a balanced classification task, and the separability assumption might not always hold. This leads to an important task-incorporating known anomalous samples into training procedures of anomaly detection models. In this work, we propose a novel model-agnostic training procedure to address this task. We reformulate one-class classification as a binary classification problem with normal data being distinguished from pseudo-anomalous samples. The pseudo-anomalous samples are drawn from low-density regions of a normalizing flow model by feeding tails of the latent distribution into the model. Such an approach allows to easily include known anomalies into the training process of an arbitrary classifier. We demonstrate that our approach shows comparable performance on one-class problems, and, most importantly, achieves comparable or superior results on tasks with variable amounts of known anomalies.

摘要

异常检测是一项具有挑战性的任务,几乎在工业和科学的所有领域都会经常出现,从欺诈检测、数据质量监测到发现罕见疾病病例以及寻找新物理现象。大多数传统的异常检测方法,如一分类支持向量机和鲁棒自编码器,都是一分类方法,专注于将正常数据与空间中的其他数据区分开来。此类方法基于正常类和异常类可分离的假设,因此没有考虑任何可用的异常样本。然而,在实际场景中,一些异常样本通常是可用的;不过,其数量通常远低于平衡分类任务所需的数量,而且可分离性假设可能并不总是成立。这就引出了一项重要任务——将已知的异常样本纳入异常检测模型的训练过程。在这项工作中,我们提出了一种新颖的与模型无关的训练过程来解决此任务。我们将一分类重新表述为一个二分类问题,将正常数据与伪异常样本区分开来。伪异常样本是通过将潜在分布的尾部输入到归一化流模型的低密度区域中抽取出来的。这种方法能够轻松地将已知异常纳入任意分类器的训练过程。我们证明,我们的方法在一分类问题上表现出可比的性能,而且最重要的是,在具有不同数量已知异常的任务上取得了可比或更优的结果。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/50b1/8627226/b5d304f4b7b0/peerj-cs-07-757-g004.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/50b1/8627226/7181927114b8/peerj-cs-07-757-g001.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/50b1/8627226/44ee37853cdb/peerj-cs-07-757-g002.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/50b1/8627226/799d7a8e7504/peerj-cs-07-757-g003.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/50b1/8627226/b5d304f4b7b0/peerj-cs-07-757-g004.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/50b1/8627226/7181927114b8/peerj-cs-07-757-g001.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/50b1/8627226/44ee37853cdb/peerj-cs-07-757-g002.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/50b1/8627226/799d7a8e7504/peerj-cs-07-757-g003.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/50b1/8627226/b5d304f4b7b0/peerj-cs-07-757-g004.jpg

相似文献

1
NFAD: fixing anomaly detection using normalizing flows.NFAD:使用归一化流修复异常检测
PeerJ Comput Sci. 2021 Nov 18;7:e757. doi: 10.7717/peerj-cs.757. eCollection 2021.
2
FRaC: a feature-modeling approach for semi-supervised and unsupervised anomaly detection.FRaC:一种用于半监督和无监督异常检测的特征建模方法。
Data Min Knowl Discov. 2012;25(1):109-133. doi: 10.1007/s10618-011-0234-x. Epub 2011 Sep 8.
3
Self-Supervised Anomaly Detection from Anomalous Training Data via Iterative Latent Token Masking.通过迭代潜在令牌掩码从异常训练数据中进行自监督异常检测。
IEEE Int Conf Comput Vis Workshops. 2023 Dec 25;2023:2394-2402. doi: 10.1109/ICCVW60793.2023.00254.
4
A machine learning approach using conditional normalizing flow to address extreme class imbalance problems in personal health records.一种使用条件归一化流来解决个人健康记录中极端类别不平衡问题的机器学习方法。
BioData Min. 2024 May 25;17(1):14. doi: 10.1186/s13040-024-00366-0.
5
An Iterative Method for Unsupervised Robust Anomaly Detection Under Data Contamination.
IEEE Trans Neural Netw Learn Syst. 2024 Oct;35(10):13327-13339. doi: 10.1109/TNNLS.2023.3267028. Epub 2024 Oct 7.
6
MSFlow: Multiscale Flow-Based Framework for Unsupervised Anomaly Detection.MSFlow:用于无监督异常检测的基于多尺度流的框架。
IEEE Trans Neural Netw Learn Syst. 2025 Feb;36(2):2437-2450. doi: 10.1109/TNNLS.2023.3344118. Epub 2025 Feb 6.
7
MOCCA: Multilayer One-Class Classification for Anomaly Detection.MOCCA:用于异常检测的多层单类分类
IEEE Trans Neural Netw Learn Syst. 2022 Jun;33(6):2313-2323. doi: 10.1109/TNNLS.2021.3130074. Epub 2022 Jun 1.
8
A Deep One-Class Neural Network for Anomalous Event Detection in Complex Scenes.用于复杂场景中异常事件检测的深度一类神经网络。
IEEE Trans Neural Netw Learn Syst. 2020 Jul;31(7):2609-2622. doi: 10.1109/TNNLS.2019.2933554. Epub 2019 Sep 5.
9
A filter-augmented auto-encoder with learnable normalization for robust multivariate time series anomaly detection.一种具有可学习归一化的滤波器增强自动编码器,用于稳健的多元时间序列异常检测。
Neural Netw. 2024 Feb;170:478-493. doi: 10.1016/j.neunet.2023.11.047. Epub 2023 Nov 27.
10
Improving Variational Autoencoders for New Physics Detection at the LHC With Normalizing Flows.利用归一化流改进用于大型强子对撞机新物理探测的变分自编码器。
Front Big Data. 2022 Feb 28;5:803685. doi: 10.3389/fdata.2022.803685. eCollection 2022.

本文引用的文献

1
Feature Encoding With Autoencoders for Weakly Supervised Anomaly Detection.用于弱监督异常检测的自动编码器特征编码
IEEE Trans Neural Netw Learn Syst. 2022 Jun;33(6):2454-2465. doi: 10.1109/TNNLS.2021.3086137. Epub 2022 Jun 1.
2
Human-level concept learning through probabilistic program induction.通过概率编程归纳实现人类水平的概念学习。
Science. 2015 Dec 11;350(6266):1332-8. doi: 10.1126/science.aab3050.
3
Searching for exotic particles in high-energy physics with deep learning.用深度学习在高能物理学中寻找奇异粒子。
Nat Commun. 2014 Jul 2;5:4308. doi: 10.1038/ncomms5308.