Department of Applied Mathematics and Statistics, Colorado School of Mines, Golden, USA.
Department of Applied Mathematics and Statistics, Department of Computer Science, Colorado School of Mines, Golden, USA.
Sci Rep. 2023 Mar 18;13(1):4501. doi: 10.1038/s41598-023-31521-y.
A normalizing flow (NF) is a mapping that transforms a chosen probability distribution to a normal distribution. Such flows are a common technique used for data generation and density estimation in machine learning and data science. The density estimate obtained with a NF requires a change of variables formula that involves the computation of the Jacobian determinant of the NF transformation. In order to tractably compute this determinant, continuous normalizing flows (CNF) estimate the mapping and its Jacobian determinant using a neural ODE. Optimal transport (OT) theory has been successfully used to assist in finding CNFs by formulating them as OT problems with a soft penalty for enforcing the standard normal distribution as a target measure. A drawback of OT-based CNFs is the addition of a hyperparameter, [Formula: see text], that controls the strength of the soft penalty and requires significant tuning. We present JKO-Flow, an algorithm to solve OT-based CNF without the need of tuning [Formula: see text]. This is achieved by integrating the OT CNF framework into a Wasserstein gradient flow framework, also known as the JKO scheme. Instead of tuning [Formula: see text], we repeatedly solve the optimization problem for a fixed [Formula: see text] effectively performing a JKO update with a time-step [Formula: see text]. Hence we obtain a "divide and conquer" algorithm by repeatedly solving simpler problems instead of solving a potentially harder problem with large [Formula: see text].
规范化流（NF）是一种将选定的概率分布转换为正态分布的映射。这种流是机器学习和数据科学中用于数据生成和密度估计的常用技术。使用 NF 获得的密度估计需要一个变量变换公式，其中涉及 NF 变换的雅可比行列式的计算。为了可计算地计算这个行列式，连续规范化流（CNF）使用神经 ODE 来估计映射及其雅可比行列式。最优传输（OT）理论已成功用于通过将其表述为具有软惩罚的 OT 问题来辅助找到 CNF，以强制将标准正态分布作为目标测度。基于 OT 的 CNF 的一个缺点是添加了一个超参数 [Formula: see text]，它控制软惩罚的强度并且需要大量调整。我们提出了 JKO-Flow，这是一种无需调整 [Formula: see text] 即可解决基于 OT 的 CNF 的算法。这是通过将 OT CNF 框架集成到 Wasserstein 梯度流框架（也称为 JKO 方案）中来实现的。我们不是调整 [Formula: see text]，而是为固定的 [Formula: see text] 重复解决优化问题，有效地使用时间步长 [Formula: see text] 执行 JKO 更新。因此，我们通过反复解决更简单的问题而不是解决具有大 [Formula: see text] 的潜在更难的问题来获得“分而治之”算法。
