- This event has passed.
LANS Seminar
November 16, 2022 @ 10:30 - 11:30 CST
Seminar Title: A Nonlocal Gradient for High-Dimensional Black-Box Optimization in Scientific Applications
Speaker: Guannan Zhang, Senior Research Scientist, Oak Ridge National Laboratory
Date/Time: November 16, 2022 / 10:30 AM – 11:30 AM
Location: See meeting URL on the cels-seminars website (requires Argonne login)
Host: Prasanna Balaprakash
Description: In this talk, we consider the problem of minimizing multi-modal loss functions with many local optima. Since the local gradient points to the direction of the steepest slope in an infinitesimal neighborhood, an optimizer guided by the local gradient is often trapped in a local minimum. To address this issue, we develop a novel nonlocal gradient to skip small local minima by capturing major structures of the loss’s landscape in black-box optimization. The nonlocal gradient is defined by a directional Gaussian smoothing (DGS) approach. The key idea is to conducts 1D long-range exploration with a large smoothing radius along orthogonal directions, each of which defines a nonlocal directional derivative as a 1D integral. Such long-range exploration enables the nonlocal gradient to skip small local minima. We use the Gauss-Hermite quadrature rule to approximate the d 1D integrals to obtain an accurate estimator. We also provide theoretical analysis on the convergence of the method on nonconvex landscape. In this work, we investigate the scenario where the objective function is composed of a convex function, perturbed by a highly oscillating, deterministic noise. We provide a convergence theory under which the iterates converge to a tightened neighborhood of the solution, whose size is characterized by the noise frequency. We complement our theoretical analysis with numerical experiments to illustrate the performance of this approach.
Bio: Dr. Guannan Zhang is a Senior Research Scientist in Machine Learning and Data Analytics Group at Oak Ridge National Laboratory (ORNL). He earned my Ph.D. in applied mathematics at Florida State University in 2012. He joined ORNL in 2012 as the Householder fellow in Computer Science and Mathematics Division. Guannan’s research interests include high-dimensional approximation, uncertainty quantification, machine learning and artificial intelligence, stochastic optimization and control, numerical solution of stochastic differential equations. He received the DOE Early Career Award in 2022. He has been holding a joint faculty appointment with the Department of Mathematics and Statistics at Auburn University since 2014.
Please note that the meeting URL for this event can be seen on the cels-seminars website, which requires an Argonne login.