Mathematics Colloquium
Thermodynamic sampling algorithms with applications to molecular dynamics and neural network parameterization
Speaker: Ben Leimkuhler, Edinburgh
Location: Warren Weaver Hall 1302
Date: Monday, October 21, 2019, 3:45 p.m.
Synopsis:
Motivated by problems in molecular dynamics, we have been investigating a wide variety of methods for sampling high dimensional distributions based on discretization of stochastic differential equations. In addition to providing high accuracy and better control of thermodynamic properties in chemical physics, these methods have the potential to impact current practice in statistical sampling and optimization, for example the Bayesian parameterization of neural networks which is the cornerstone of much recent research in deep learning. I will focus on the adaptive Langevin (AdL) method [1,2,3], which automatically corrects for gradient noise in Langevin simulations. We have combined this with a partitioning based on neural network structure to design an optimizer (AdLaLa) for use in machine learning and have tested it for certain forms of binary classification [4]. I will discuss numerical tests of the convergence this method in comparison with popular alternatives such as stochastic gradient descent (SGD) and stochastic gradient Langevin dynamics (SGLD).
[Quoted text hidden]