Constrained Maximum Likelihood Estimation by EM-Type Methods

This paper proposes EM-type algorithms for maximum likelihood (ML) estimation under liner equality and inequality constraints. Two algorithms of generalized gradient projection (GP) and generalized conjugate gradient projection (CGP) are developed. EM algorithm is a popular method for unconstrained estimation. Using GP and CGP two algorithms that extend EM for constrained estimation are proposed; these algorithms are called GP-EM and CGP-EM. Two previously proposed EM-type methods for constrained ML estimation are described, and it is shown that their corresponding published convergence results are incorrect. It’s discussed that the GP-EM and the CGP-EM algorithms are widely applicable. As an example, application of GP-EM to three problems in the area of estimation of covariance from incomplete data is considered, and their speed of convergence is compared. With little additional implementation complexity over GP-EM, the CGP-EM algorithm converges noticably faster than the GP-EM. Among other extensions of GP and CGP to constrained ML estimation, extension of the Fisher scoring algorithm by GP and CGP to handle constraints is pointed out.

Keywords:

EM algorithm , Fisher scoring , Linear equality constraints , Linear inequality constraints , Test of hypothesis , Profile likelihood.