On EM algorithms and their proximal generalizations
ESAIM: Probability and Statistics, Tome 12 (2008), pp. 308-326

In this paper, we analyze the celebrated EM algorithm from the point of view of proximal point algorithms. More precisely, we study a new type of generalization of the EM procedure introduced in [Chretien and Hero (1998)] and called Kullback-proximal algorithms. The proximal framework allows us to prove new results concerning the cluster points. An essential contribution is a detailed analysis of the case where some cluster points lie on the boundary of the parameter space.

DOI : 10.1051/ps:2007041
Classification : 65C20, 65C60
Keywords: maximum likelihood estimation (MLE), EM algorithm, proximal point algorithm, Karush-Kuhn-Tucker condition, mixture densities, competing risks models 
@article{PS_2008__12__308_0,
     author = {Chr\'etien, St\'ephane and Hero, Alfred O.},
     title = {On {EM} algorithms and their proximal generalizations},
     journal = {ESAIM: Probability and Statistics},
     pages = {308--326},
     year = {2008},
     publisher = {EDP Sciences},
     volume = {12},
     doi = {10.1051/ps:2007041},
     mrnumber = {2404033},
     language = {en},
     url = {https://www.numdam.org/articles/10.1051/ps:2007041/}
}
TY  - JOUR
AU  - Chrétien, Stéphane
AU  - Hero, Alfred O.
TI  - On EM algorithms and their proximal generalizations
JO  - ESAIM: Probability and Statistics
PY  - 2008
SP  - 308
EP  - 326
VL  - 12
PB  - EDP Sciences
UR  - https://www.numdam.org/articles/10.1051/ps:2007041/
DO  - 10.1051/ps:2007041
LA  - en
ID  - PS_2008__12__308_0
ER  - 
%0 Journal Article
%A Chrétien, Stéphane
%A Hero, Alfred O.
%T On EM algorithms and their proximal generalizations
%J ESAIM: Probability and Statistics
%D 2008
%P 308-326
%V 12
%I EDP Sciences
%U https://www.numdam.org/articles/10.1051/ps:2007041/
%R 10.1051/ps:2007041
%G en
%F PS_2008__12__308_0
Chrétien, Stéphane; Hero, Alfred O. On EM algorithms and their proximal generalizations. ESAIM: Probability and Statistics, Tome 12 (2008), pp. 308-326. doi: 10.1051/ps:2007041

[1] H. Ahn, H. Moon and R.L. Kodell, Attribution of tumour lethality and estimation of the time to onset of occult tumours in the absence of cause-of-death information. J. Roy. Statist. Soc. Ser. C 49 (2000) 157-169. | Zbl | MR

[2] M.J. Box, A new method of constrained optimization and a comparison with other methods. Comp. J. 8 (1965) 42-52. | Zbl | MR

[3] G. Celeux, S. Chretien, F. Forbes and A. Mkhadri, A component-wise EM algorithm for mixtures. J. Comput. Graph. Statist. 10 (2001), 697-712 and INRIA RR-3746, Aug. 1999. | MR

[4] S. Chretien and A.O. Hero, Acceleration of the EM algorithm via proximal point iterations, in Proceedings of the International Symposium on Information Theory, MIT, Cambridge (1998) 444.

[5] S. Chrétien and A. Hero, Kullback proximal algorithms for maximum-likelihood estimation. IEEE Trans. Inform. Theory 46 (2000) 1800-1810. | Zbl | MR

[6] I. Csiszár, Information-type measures of divergence of probability distributions and indirect observations. Studia Sci. Math. Hung. 2 (1967) 299-318. | Zbl | MR

[7] A.P. Dempster, N.M. Laird and D.B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc., Ser. B 39 (1977) 1-38. | Zbl | MR

[8] I.A. Ibragimov and R.Z. Has'Minskii, Statistical estimation: Asymptotic theory. Springer-Verlag, New York (1981). | Zbl | MR

[9] Journal of Statistical Planning and Inference No. 107 (2002) 1-2.

[10] A.T. Kalai and S. Vempala, Simulated annealing for convex optimization. Math. Oper. Res. 31 (2006) 253-266. | MR

[11] B. Martinet, Régularisation d'inéquation variationnelles par approximations successives. Revue Francaise d'Informatique et de Recherche Operationnelle 3 (1970) 154-179. | Zbl | MR | Numdam

[12] G.J. Mclachlan and T. Krishnan, The EM algorithm and extensions, Wiley Series in Probability and Statistics: Applied Probability and Statistics. John Wiley and Sons, Inc., New York (1997). | Zbl | MR

[13] H. Moon, H. Ahn, R. Kodell and B. Pearce, A comparison of a mixture likelihood method and the EM algorithm for an estimation problme in animal carcinogenicity studies. Comput. Statist. Data Anal. 31 (1999) 227-238. | Zbl

[14] A.M. Ostrowski, Solution of equations and systems of equations. Pure and Applied Mathematics, Vol. IX. Academic Press, New York-London (1966). | Zbl | MR

[15] R.T. Rockafellar, Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14 (1976) 877-898. | Zbl | MR

[16] M. Teboulle, Entropic proximal mappings with application to nonlinear programming. Math. Oper. Res. 17 (1992) 670-690. | Zbl | MR

[17] P. Tseng, An analysis of the EM algorithm and entropy-like proximal point methods. Math. Oper. Res. 29 (2004) 27-44. | Zbl | MR

[18] C.F.J. Wu, On the convergence properties of the EM algorithm. Ann. Stat. 11 (1983) 95-103. | Zbl | MR

[19] Z.B. Zabinsky, Stochastic adaptive search for global optimization. Nonconvex Optimization and its Applications 72. Kluwer Academic Publishers, Boston, MA (2003). | Zbl | MR

[20] W.I. Zangwill and B. Mond, Nonlinear programming: a unified approach. Prentice-Hall International Series in Management. Prentice-Hall, Inc., Englewood Cliffs, N.J. (1969). | Zbl | MR

Cité par Sources :