Abstract:

Deep generative models are probabilistic generative models where the generator is parameterized by a deep neural network.  They are popular models for modeling high-dimensional data such as texts, images and speeches, and have achieved impressive empirical success. Despite demonstrated success in  empirical performance,  theoretical understanding of such models  is largely lacking . We investigate statistical properties of deep generative models from a nonparametric distribution estimation viewpoint. In the considered model, data are assumed to be observed in some high-dimensional ambient space but concentrate around some low-dimensional structure such as a lower-dimensional manifold. Estimating the distribution supported on this low-dimensional structure is challenging due to its singularity with respect to the Lebesgue measure in the ambient space. We obtain convergence rates with respect to the Wasserstein metric of  distribution estimators based on two methods: a sieve MLE based on the perturbed data and a GAN type estimator. Such an analysis  provides  insights into  i) how deep generative models can avoid the curse of dimensionality and outperform classical nonparametric estimates,  and  ii) how likelihood approaches work for singular distribution estimation, especially in adapting to the intrinsic geometry of the data. 

Short Bio: 

Lizhen Lin is a professor of statistics in the Department of Mathematics at the University of Maryland where she currently also serves as the director of the statistics program.  Her areas of expertise are in Bayesian modeling and theory for high-dimensional and infinite-dimensional models, statistics on manifolds and statistical network analysis. Her recent effort has been on understanding statistical properties of deep neural network models in particular deep generative models.