Journal article
Relative entropy minimization over Hilbert spaces via Robbins-Monro
AIMS mathematics, v 4(3), pp 359-383
01 Jan 2019
Featured in Collection : UN Sustainable Development Goals @ Drexel
Abstract
One way of getting insight into non-Gaussian measures is to first obtain good Gaussian approximations. These best fit Gaussians can then provide a sense of the mean and variance of the distribution of interest. They can also be used to accelerate sampling algorithms. This begs the question of how one should measure optimality, and how to then obtain this optimal approximation. Here, we consider the problem of minimizing the distance between a family of Gaussians and the target measure with respect to relative entropy, or Kullback-Leibler divergence. As we are interested in applications in the infinite dimensional setting, it is desirable to have convergent algorithms that are well posed on abstract Hilbert spaces. We examine this minimization problem by seeking roots of the first variation of relative entropy, taken with respect to the mean of the Gaussian, leaving the covariance fixed. We prove the convergence of Robbins-Monro type root finding algorithms in this context, highlighting the assumptions necessary for convergence to relative entropy minimizers. Numerical examples are included to illustrate the algorithms.
Metrics
12 Record Views
Details
- Title
- Relative entropy minimization over Hilbert spaces via Robbins-Monro
- Creators
- Gideon Simpson - Drexel UniversityDaniel Watkins - Oregon State University
- Publication Details
- AIMS mathematics, v 4(3), pp 359-383
- Publisher
- Amer Inst Mathematical Sciences-Aims
- Number of pages
- 25
- Grant note
- DE-SC0012733 / US Department of Energy; United States Department of Energy (DOE) DMS-1818716 / US National Science Foundation; National Science Foundation (NSF)
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- Mathematics
- Web of Science ID
- WOS:000478768800002
- Scopus ID
- 2-s2.0-85073391746
- Other Identifier
- 991019169688404721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
Source: SDGs in the Output
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- Web of Science research areas
- Mathematics
- Mathematics, Applied