Wu Zhenqing, Gong Wenwu, Yu Ziying, Zhu Liuxin, Yang Lili
Department of Statistics and Data Science, Southern University of Science and Technology, Shenzhen, 518055, China.
Shenzhen Key Laboratory of Safety and Security for Next Generation of Industrial Internet, Shenzhen, 518055, China.
Sci Rep. 2025 May 15;15(1):16914. doi: 10.1038/s41598-025-01735-3.
We propose the replica exchange adaptively weighted stochastic gradient Langevin dynamics (REAWSGLD) algorithm, designed explicitly for Bayesian learning with complex energy landscapes encountered in big data problems. By merging the 1/k-ensemble and replica exchange methods, this algorithm effectively escapes local traps in Monte Carlo simulation and non-convex optimization. It operates by running two Langevin dynamics processes concurrently at different temperatures, enabling position swaps between them. The lower temperature process, influenced by the 1/k-ensemble method, focuses on exploiting local geometry by protruding low-energy regions and biasing the sampling towards them. Meanwhile, the higher temperature process, influenced by larger noises, facilitates global exploration across the entire domain. The 1/k-ensemble and replica exchange methods are complementary: the 1/k-ensemble method mitigates the risk of the replica exchange method excessively exploring distribution tails, while the replica exchange method enhances the global exploration capability of the 1/k-ensemble method. The proposed algorithm has been empirically evaluated across various experiments, demonstrating its efficacy in navigating complex energy landscapes. The numerical results highlight its potential for Monte Carlo simulation and non-convex optimization in contemporary machine learning tasks.
我们提出了复制交换自适应加权随机梯度朗之万动力学（REAWSGLD）算法，该算法专为处理大数据问题中遇到的具有复杂能量景观的贝叶斯学习而设计。通过合并1/k系综和复制交换方法，该算法有效地避免了蒙特卡罗模拟和非凸优化中的局部陷阱。它通过在不同温度下同时运行两个朗之万动力学过程来操作，允许它们之间进行位置交换。受1/k系综方法影响的较低温度过程，通过突出低能量区域并将采样偏向这些区域来专注于利用局部几何结构。同时，受较大噪声影响的较高温度过程有助于在整个域中进行全局探索。1/k系综和复制交换方法是互补的：1/k系综方法减轻了复制交换方法过度探索分布尾部的风险，而复制交换方法增强了1/k系综方法的全局探索能力。所提出的算法已在各种实验中进行了实证评估，证明了其在穿越复杂能量景观方面的有效性。数值结果突出了其在当代机器学习任务中的蒙特卡罗模拟和非凸优化潜力。
