Skip directly to content

Minimize RSR Award Detail

Research Spending & Results

Award Detail

Doing Business As Name:Yale University
  • Huibin Zhou
  • (203) 432-2460
Award Date:06/14/2021
Estimated Total Award Amount: $ 370,000
Funds Obligated to Date: $ 370,000
  • FY 2021=$370,000
Start Date:07/01/2021
End Date:06/30/2024
Transaction Type:Grant
Awarding Agency Code:4900
Funding Agency Code:4900
CFDA Number:47.049
Primary Program Source:040100 NSF RESEARCH & RELATED ACTIVIT
Award Title or Description:Overparameterization, Global Convergence of the Expectation-Maximization Algorithm, and Beyond
Federal Award ID Number:2112918
DUNS ID:043207562
Parent DUNS ID:043207562
Program Officer:
  • Yong Zeng
  • (703) 292-7902

Awardee Location

Street:Office of Sponsored Projects
City:New Haven
County:New Haven
Awardee Cong. District:03

Primary Place of Performance

Organization Name:Yale University
Street:10 Hillhouse
City:New Haven
County:New Haven
Cong. District:03

Abstract at Time of Award

The expectation-maximization (EM) algorithm is among the most popular algorithms for statistical inference. Despite a wide range of successful applications in both statistics and machine learning, there is little finite-sample theoretical analysis explaining the effectiveness of EM and its variants. Recently, there have been some encouraging successes on the global convergence guarantee of the EM algorithm, but often under unrealistic and impractical assumptions. The PI will integrate the recent success of overparametrization in deep learning with EM to overcome the aforementioned limitations. The research presented in this project will significantly advance the celebrated algorithms in statistics and machine learning including EM, mean-field variational inference, and Gibbs sampling by providing guarantees of global convergence and statistical optimalities. The research will help address the non-convex optimization challenges for a range of important and classical statistical models and shed light on the recent successes of deep learning. The wide range of applications of EM, mean-field variational inference, and Gibbs sampling and the importance of clustering ensure that the progress we make towards our objectives will have a great impact on the broad scientific community which includes neuroscience and medicine. Research results from this project will be disseminated through research articles, workshops, and seminar series to researchers in other disciplines. The project will integrate research and education by teaching monograph courses and organizing workshops and seminars to support graduate students and postdocs, particularly women, underrepresented minorities, domestic students, and young researchers, to work on this topic. The PI will develop methods for obtaining global convergence under possibly the weakest assumptions for a general class of latent variable models’ estimation with an unknown number of clusters. The PI will address the following questions: 1) can we show that the overparameterized EM converges globally to the true parameters without any separation condition and any knowledge of the number of clusters and cluster sizes under a certain distance (such as Wasserstein)? 2) how fast does the algorithm converge? 3) what are the parameter estimation and clustering error rates and how do they compare to the optimal statistical accuracy? and 4) if not optimal statistically, can we achieve the optimality by adding a second stage EM initialized by the output of the overparameterized EM? There are three aims to develop a comprehensive theory to analyze the overparameterized EM and go beyond: 1) studying the global convergence of overparameterized EM for Gaussian Mixtures for both parameter estimation and latent cluster recovery and statistical optimality of the two-stage EM, 2) extending the two-stage EM to its variants including two-stage mean-field variational inference and Gibbs sampling and considering a unified analysis for a class of overparameterized algorithms, and 3) extending the analysis for Gaussian mixtures to general location mixture models and Stochastic Block Models and possibly a unified framework of latent variable models. In addition, the PI will work closely with the Yale Child Study Center and Yale Therapeutic Radiology Department to explore the appropriate EM algorithm and its variants for neuroscience, autism spectrum disorder, and cancer risk stratification. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

For specific questions or comments about this information including the NSF Project Outcomes Report, contact us.