In this paper, we analyze the celebrated EM algorithm from the point of view of proximal point algorithms. More precisely, we study a new type of generalization of the EM procedure introduced in [Chretien and Hero (1998)] and called Kullback-proximal algorithms. The proximal framework allows us to prove new results concerning the cluster points. An essential contribution is a detailed analysis of the case where some cluster points lie on the boundary of the parameter space.

Keywords: maximum likelihood estimation (MLE), EM algorithm, proximal point algorithm, Karush-Kuhn-Tucker condition, mixture densities, competing risks models

@article{PS_2008__12__308_0, author = {Chr\'etien, St\'ephane and Hero, Alfred O.}, title = {On {EM} algorithms and their proximal generalizations}, journal = {ESAIM: Probability and Statistics}, pages = {308--326}, publisher = {EDP-Sciences}, volume = {12}, year = {2008}, doi = {10.1051/ps:2007041}, mrnumber = {2404033}, language = {en}, url = {http://www.numdam.org/articles/10.1051/ps:2007041/} }

TY - JOUR AU - Chrétien, Stéphane AU - Hero, Alfred O. TI - On EM algorithms and their proximal generalizations JO - ESAIM: Probability and Statistics PY - 2008 SP - 308 EP - 326 VL - 12 PB - EDP-Sciences UR - http://www.numdam.org/articles/10.1051/ps:2007041/ DO - 10.1051/ps:2007041 LA - en ID - PS_2008__12__308_0 ER -

Chrétien, Stéphane; Hero, Alfred O. On EM algorithms and their proximal generalizations. ESAIM: Probability and Statistics, Volume 12 (2008), pp. 308-326. doi : 10.1051/ps:2007041. http://www.numdam.org/articles/10.1051/ps:2007041/

[1] Attribution of tumour lethality and estimation of the time to onset of occult tumours in the absence of cause-of-death information. J. Roy. Statist. Soc. Ser. C 49 (2000) 157-169. | MR | Zbl

, and ,[2] A new method of constrained optimization and a comparison with other methods. Comp. J. 8 (1965) 42-52. | MR | Zbl

,[3] A component-wise EM algorithm for mixtures. J. Comput. Graph. Statist. 10 (2001), 697-712 and INRIA RR-3746, Aug. 1999. | MR

, , and ,[4] Acceleration of the EM algorithm via proximal point iterations, in Proceedings of the International Symposium on Information Theory, MIT, Cambridge (1998) 444.

and ,[5] Kullback proximal algorithms for maximum-likelihood estimation. IEEE Trans. Inform. Theory 46 (2000) 1800-1810. | MR | Zbl

and ,[6] Information-type measures of divergence of probability distributions and indirect observations. Studia Sci. Math. Hung. 2 (1967) 299-318. | MR | Zbl

,[7] Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc., Ser. B 39 (1977) 1-38. | MR | Zbl

, and ,[8] Statistical estimation: Asymptotic theory. Springer-Verlag, New York (1981). | MR | Zbl

and ,[9] Journal of Statistical Planning and Inference No. 107 (2002) 1-2.

[10] Simulated annealing for convex optimization. Math. Oper. Res. 31 (2006) 253-266. | MR

and ,[11] Régularisation d'inéquation variationnelles par approximations successives. Revue Francaise d'Informatique et de Recherche Operationnelle 3 (1970) 154-179. | Numdam | MR | Zbl

,[12] The EM algorithm and extensions, Wiley Series in Probability and Statistics: Applied Probability and Statistics. John Wiley and Sons, Inc., New York (1997). | MR | Zbl

and ,[13] A comparison of a mixture likelihood method and the EM algorithm for an estimation problme in animal carcinogenicity studies. Comput. Statist. Data Anal. 31 (1999) 227-238. | Zbl

, , and ,[14] Solution of equations and systems of equations. Pure and Applied Mathematics, Vol. IX. Academic Press, New York-London (1966). | MR | Zbl

,[15] Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14 (1976) 877-898. | MR | Zbl

,[16] Entropic proximal mappings with application to nonlinear programming. Math. Oper. Res. 17 (1992) 670-690. | MR | Zbl

,[17] An analysis of the EM algorithm and entropy-like proximal point methods. Math. Oper. Res. 29 (2004) 27-44. | MR | Zbl

,[18] On the convergence properties of the EM algorithm. Ann. Stat. 11 (1983) 95-103. | MR | Zbl

,[19] Stochastic adaptive search for global optimization. Nonconvex Optimization and its Applications 72. Kluwer Academic Publishers, Boston, MA (2003). | MR | Zbl

,[20] Nonlinear programming: a unified approach. Prentice-Hall International Series in Management. Prentice-Hall, Inc., Englewood Cliffs, N.J. (1969). | MR | Zbl

and ,*Cited by Sources: *