Wuchen Li: "Accelerated Information Gradient Flow"

High Dimensional Hamilton-Jacobi PDEs 2020
Workshop II: PDE and Inverse Problem Methods in Machine Learning
"Accelerated Information Gradient Flow"
Wuchen Li - University of California, Los Angeles (UCLA)
Abstract: We present a systematic framework for the Nesterov's accelerated gradient flows in the spaces of probabilities embedded with information metrics. Here two metrics are considered, including both the Fisher-Rao metric and the Wasserstein-2 metric. For the Wasserstein-2 metric case, we prove the convergence properties of the accelerated gradient flows, and introduce their formulations in Gaussian families. Furthermore, we propose a practical discrete-time algorithm in particle implementations with an adaptive restart technique. We formulate a novel bandwidth selection method, which learns the Wasserstein-2 gradient direction from Brownian-motion samples. Experimental results including Bayesian inference show the strength of the current method compared with the state-of-the-art. Further connections with inverse problems and data related optimization techniques will be discussed.
Institute for Pure and Applied Mathematics, UCLA
April 23, 2020
For more information: www.ipam.ucla.edu/hjws2