DMS Statistics and Data Science Seminar

Speaker:  Dr. Yuting Wei (the Wharton School, University of Pennsylvania)

Title: Towards faster non-asymptotic convergence for diffusion-based generative models

Abstract: Diffusion models, which convert noise into new data instances by learning to reverse a Markov diffusion process, have become a cornerstone in contemporary generative artificial intelligence. While their practical power has now been widely recognized, the theoretical underpinnings remain far from mature.  In this work, we develop a suite of non-asymptotic theory towards understanding the data generation process of diffusion models in discrete time, assuming access to \(\ell_2\)-accurate estimates of the (Stein) score functions. For a popular deterministic sampler (based on the probability flow ODE), we establish a convergence rate proportional to \(1/T\) (with \(T\) the total number of steps), improving upon past results; for another mainstream stochastic sampler (i.e., a type of the denoising diffusion probabilistic model), we derive a convergence rate proportional to \(1/\sqrt{T}\), matching the state-of-the-art theory. Imposing only minimal assumptions on the target data distribution (e.g., no smoothness assumption is imposed), our results characterize how \(\ell_2\) score estimation errors affect the quality of the data generation processes. Further, we design two accelerated variants, improving the convergence to \(1/T^2\) for the ODE-based sampler and \(1/T\) for the DDPM-type sampler, which might be of independent theoretical and empirical interest.

Bio: Dr. Yuting Wei is currently an assistant professor in the Statistics and Data Science Department at the Wharton School, University of Pennsylvania. Prior to that, Dr. Wei spent two years at Carnegie Mellon University as an assistant professor and one year at Stanford University as a Stein Fellow. She received her Ph.D. in statistics at the University of California, Berkeley. She was the recipient of the 2023 Google Research Scholar Award, 2022 NSF Career award, and the Erich L. Lehmann Citation from the Berkeley statistics department. Her research interests include high-dimensional and non-parametric statistics, statistical machine learning, and reinforcement learning.