On EM algorithms and their proximal generalizations
ESAIM: Probability and Statistics, Tome 12 (2008), pp. 308-326.

In this paper, we analyze the celebrated EM algorithm from the point of view of proximal point algorithms. More precisely, we study a new type of generalization of the EM procedure introduced in [Chretien and Hero (1998)] and called Kullback-proximal algorithms. The proximal framework allows us to prove new results concerning the cluster points. An essential contribution is a detailed analysis of the case where some cluster points lie on the boundary of the parameter space.

DOI : https://doi.org/10.1051/ps:2007041
Classification : 65C20,  65C60
Mots clés : maximum likelihood estimation (MLE), EM algorithm, proximal point algorithm, Karush-Kuhn-Tucker condition, mixture densities, competing risks models
@article{PS_2008__12__308_0,
     author = {Chr\'etien, St\'ephane and Hero, Alfred O.},
     title = {On {EM} algorithms and their proximal generalizations},
     journal = {ESAIM: Probability and Statistics},
     pages = {308--326},
     publisher = {EDP-Sciences},
     volume = {12},
     year = {2008},
     doi = {10.1051/ps:2007041},
     mrnumber = {2404033},
     language = {en},
     url = {http://www.numdam.org/articles/10.1051/ps:2007041/}
}
TY  - JOUR
AU  - Chrétien, Stéphane
AU  - Hero, Alfred O.
TI  - On EM algorithms and their proximal generalizations
JO  - ESAIM: Probability and Statistics
PY  - 2008
DA  - 2008///
SP  - 308
EP  - 326
VL  - 12
PB  - EDP-Sciences
UR  - http://www.numdam.org/articles/10.1051/ps:2007041/
UR  - https://www.ams.org/mathscinet-getitem?mr=2404033
UR  - https://doi.org/10.1051/ps:2007041
DO  - 10.1051/ps:2007041
LA  - en
ID  - PS_2008__12__308_0
ER  - 
Chrétien, Stéphane; Hero, Alfred O. On EM algorithms and their proximal generalizations. ESAIM: Probability and Statistics, Tome 12 (2008), pp. 308-326. doi : 10.1051/ps:2007041. http://www.numdam.org/articles/10.1051/ps:2007041/

[1] H. Ahn, H. Moon and R.L. Kodell, Attribution of tumour lethality and estimation of the time to onset of occult tumours in the absence of cause-of-death information. J. Roy. Statist. Soc. Ser. C 49 (2000) 157-169. | MR 1821319 | Zbl 0944.62097

[2] M.J. Box, A new method of constrained optimization and a comparison with other methods. Comp. J. 8 (1965) 42-52. | MR 184734 | Zbl 0142.11305

[3] G. Celeux, S. Chretien, F. Forbes and A. Mkhadri, A component-wise EM algorithm for mixtures. J. Comput. Graph. Statist. 10 (2001), 697-712 and INRIA RR-3746, Aug. 1999. | MR 1938975

[4] S. Chretien and A.O. Hero, Acceleration of the EM algorithm via proximal point iterations, in Proceedings of the International Symposium on Information Theory, MIT, Cambridge (1998) 444.

[5] S. Chrétien and A. Hero, Kullback proximal algorithms for maximum-likelihood estimation. IEEE Trans. Inform. Theory 46 (2000) 1800-1810. | MR 1790321 | Zbl 1021.68101

[6] I. Csiszár, Information-type measures of divergence of probability distributions and indirect observations. Studia Sci. Math. Hung. 2 (1967) 299-318. | MR 219345 | Zbl 0157.25802

[7] A.P. Dempster, N.M. Laird and D.B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc., Ser. B 39 (1977) 1-38. | MR 501537 | Zbl 0364.62022

[8] I.A. Ibragimov and R.Z. Has'Minskii, Statistical estimation: Asymptotic theory. Springer-Verlag, New York (1981). | MR 620321 | Zbl 0467.62026

[9] Journal of Statistical Planning and Inference No. 107 (2002) 1-2.

[10] A.T. Kalai and S. Vempala, Simulated annealing for convex optimization. Math. Oper. Res. 31 (2006) 253-266. | MR 2233996

[11] B. Martinet, Régularisation d'inéquation variationnelles par approximations successives. Revue Francaise d'Informatique et de Recherche Operationnelle 3 (1970) 154-179. | Numdam | MR 298899 | Zbl 0215.21103

[12] G.J. Mclachlan and T. Krishnan, The EM algorithm and extensions, Wiley Series in Probability and Statistics: Applied Probability and Statistics. John Wiley and Sons, Inc., New York (1997). | MR 1417721 | Zbl 0882.62012

[13] H. Moon, H. Ahn, R. Kodell and B. Pearce, A comparison of a mixture likelihood method and the EM algorithm for an estimation problme in animal carcinogenicity studies. Comput. Statist. Data Anal. 31 (1999) 227-238. | Zbl 0935.62125

[14] A.M. Ostrowski, Solution of equations and systems of equations. Pure and Applied Mathematics, Vol. IX. Academic Press, New York-London (1966). | MR 216746 | Zbl 0222.65070

[15] R.T. Rockafellar, Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14 (1976) 877-898. | MR 410483 | Zbl 0358.90053

[16] M. Teboulle, Entropic proximal mappings with application to nonlinear programming. Math. Oper. Res. 17 (1992) 670-690. | MR 1177730 | Zbl 0766.90071

[17] P. Tseng, An analysis of the EM algorithm and entropy-like proximal point methods. Math. Oper. Res. 29 (2004) 27-44. | MR 2065712 | Zbl 1082.90092

[18] C.F.J. Wu, On the convergence properties of the EM algorithm. Ann. Stat. 11 (1983) 95-103. | MR 684867 | Zbl 0517.62035

[19] Z.B. Zabinsky, Stochastic adaptive search for global optimization. Nonconvex Optimization and its Applications 72. Kluwer Academic Publishers, Boston, MA (2003). | MR 2015270 | Zbl 1044.90001

[20] W.I. Zangwill and B. Mond, Nonlinear programming: a unified approach. Prentice-Hall International Series in Management. Prentice-Hall, Inc., Englewood Cliffs, N.J. (1969). | MR 359816 | Zbl 0195.20804

Cité par Sources :