Qiu Chen, Mandt Stephan, Rudolph Maja
Bosch Center for AI, 71272 Renningen, Germany.
Department of Computer Science, TU Kaiserslautern, 67653 Kaiserslautern, Germany.
Entropy (Basel). 2021 Nov 24;23(12):1563. doi: 10.3390/e23121563.
Deep probabilistic time series forecasting models have become an integral part of machine learning. While several powerful generative models have been proposed, we provide evidence that their associated inference models are oftentimes too limited and cause the generative model to predict mode-averaged dynamics. Mode-averaging is problematic since many real-world sequences are highly multi-modal, and their averaged dynamics are unphysical (e.g., predicted taxi trajectories might run through buildings on the street map). To better capture multi-modality, we develop variational dynamic mixtures (VDM): a new variational family to infer sequential latent variables. The VDM approximate posterior at each time step is a mixture density network, whose parameters come from propagating multiple samples through a recurrent architecture. This results in an expressive multi-modal posterior approximation. In an empirical study, we show that VDM outperforms competing approaches on highly multi-modal datasets from different domains.
深度概率时间序列预测模型已成为机器学习不可或缺的一部分。虽然已经提出了几种强大的生成模型，但我们证明，它们相关的推理模型往往过于有限，导致生成模型预测模式平均动态。模式平均存在问题，因为许多现实世界的序列是高度多模态的，其平均动态是不符合实际的（例如，预测的出租车轨迹可能会穿过街道地图上的建筑物）。为了更好地捕捉多模态，我们开发了变分动态混合模型（VDM）：一种用于推断序列潜在变量的新变分族。每个时间步的VDM近似后验是一个混合密度网络，其参数来自通过循环架构传播多个样本。这导致了一个富有表现力的多模态后验近似。在一项实证研究中，我们表明，在来自不同领域的高度多模态数据集上，VDM优于竞争方法。
