The Expectation-Maximization Algorithm (EM) is a widely used method allowing to estimate the maximum likelihood of models involving latent variables. When the Expectation step cannot be computed easily, one can use stochastic versions of the EM such as the Stochastic Approximation EM. This algorithm, however, has the drawback to require the joint likelihood to belong to the curved exponential family. To overcome this problem, [16] introduced a rewriting of the model which “exponentializes” it by considering the parameter as an additional latent variable following a Normal distribution centered on the newly defined parameters and with fixed variance. The likelihood of this new exponentialized model now belongs to the curved exponential family. Although often used, there is no guarantee that the estimated mean is close to the maximum likelihood estimate of the initial model. In this paper, we quantify the error done in this estimation while considering the exponentialized model instead of the initial one. By verifying those results on an example, we see that a trade-off must be made between the speed of convergence and the tolerated error. Finally, we propose a new algorithm allowing a better estimation of the parameter in a reasonable computation time to reduce the bias.
Keywords: Expectation maximization, curved exponential, mixed effect models, convergence analysis
@article{PS_2021__25_1_408_0,
author = {Debavelaere, Vianney and Allassonni\`ere, St\'ephanie},
title = {On the curved exponential family in the {Stochastic} {Approximation} {Expectation} {Maximization} {Algorithm}},
journal = {ESAIM: Probability and Statistics},
pages = {408--432},
year = {2021},
publisher = {EDP-Sciences},
volume = {25},
doi = {10.1051/ps/2021015},
mrnumber = {4320884},
zbl = {1493.62114},
language = {en},
url = {https://www.numdam.org/articles/10.1051/ps/2021015/}
}
TY - JOUR AU - Debavelaere, Vianney AU - Allassonnière, Stéphanie TI - On the curved exponential family in the Stochastic Approximation Expectation Maximization Algorithm JO - ESAIM: Probability and Statistics PY - 2021 SP - 408 EP - 432 VL - 25 PB - EDP-Sciences UR - https://www.numdam.org/articles/10.1051/ps/2021015/ DO - 10.1051/ps/2021015 LA - en ID - PS_2021__25_1_408_0 ER -
%0 Journal Article %A Debavelaere, Vianney %A Allassonnière, Stéphanie %T On the curved exponential family in the Stochastic Approximation Expectation Maximization Algorithm %J ESAIM: Probability and Statistics %D 2021 %P 408-432 %V 25 %I EDP-Sciences %U https://www.numdam.org/articles/10.1051/ps/2021015/ %R 10.1051/ps/2021015 %G en %F PS_2021__25_1_408_0
Debavelaere, Vianney; Allassonnière, Stéphanie. On the curved exponential family in the Stochastic Approximation Expectation Maximization Algorithm. ESAIM: Probability and Statistics, Tome 25 (2021), pp. 408-432. doi: 10.1051/ps/2021015
[1] , and , Convergent stochastic algorithm for parameter estimation in frailty models using integrated partial likelihood. Preprint (2019). | arXiv
[2] and , A new class of em algorithms. escaping local minima and handling intractable sampling. Preprint (2019). | MR | Zbl
[3] , , , et al., Construction of Bayesian deformable models via a stochastic approximation algorithm: a convergence study. Bernoulli 16 (2010) 641–678. | MR | Zbl | DOI
[4] , , et al., A stochastic algorithm for probabilistic independent component analysis. Ann. Appl. Stat. 6 (2012) 125–160. | MR | Zbl | DOI
[5] , and , Stability of stochastic approximation under verifiable conditions. SIAM J. Control Optim. 44 (2005) 283–312. | MR | Zbl | DOI
[6] , , , et al., Statistical guarantees for the em algorithm: from population to sample-based analysis. Ann. Stat. 45 (2017) 77–120. | MR | Zbl | DOI
[7] , , , , , and , Classical mathematical models for description and prediction of experimental tumor growth. PLoS Comput Biol. 10 (2014) e1003800. | DOI
[8] , and , Learning distributions of shape trajectories from longitudinal datasets: a hierarchical model on a manifold of diffeomorphisms, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018) 9271–9280.
[9] and , On em algorithms and their proximal generalizations. ESAIM: Probab. Stat. 12 (2008) 308–326. | MR | Zbl | Numdam | DOI
[10] , and , Learning the clustering of longitudinal shape data sets into a mixture of independent or branching trajectories. Int. J. Comput. Vision (2020) 1–16. | MR | Zbl
[11] , and , Convergence of a stochastic approximation version of the em algorithm. Ann. Stat. (1999) 94–128. | MR | Zbl
[12] , and , Maximum likelihood from incomplete data via the em algorithm. J. Roy. Stat. Soc. Ser. B (Methodological) 39 (1977) 1–22. | MR | Zbl
[13] , , , and , Model-based analyses of bioequivalence crossover trials using the stochastic approximation expectation maximisation algorithm. Stat. Med. 30 (2011) 2582–2600. | MR | DOI
[14] and , Second-phase hepatitis c virus RNA decline during Telaprevir-based therapy increases with drug effectiveness: implications for treatment duration. Hepatology 53 (2011) 1801–1808. | DOI
[15] and , Coupling a stochastic approximation version of em with an MCMC procedure. ESAIM: Probab. Stat. 8 (2004) 115–131. | MR | Zbl | Numdam | DOI
[16] and , Maximum likelihood estimation in nonlinear mixed effects models. Comput. Stat. Data Anal. 49 (2005) 1020–1038. | MR | Zbl | DOI
[17] , and , Properties of the stochastic approximation em algorithm with mini-batch sampling. Stat. Comput. 30 (2020) 1725–1739. | MR | Zbl | DOI
[18] , and , Deterministic approximate em algorithm; application to the Riemann approximation em and the tempered em. Preprint (2020). | arXiv
[19] , Mixed effects models for the population approach: models, tasks, methods and tools. CRC Press (2014). | MR | Zbl | DOI
[20] and , Estimation of population pharmacokinetic parameters of saquinavir in HIV patients with the Monolix software. J. Pharmacokinet. Pharmacodyn. 34 (2007) 229–249. | DOI
[21] , An efficient stochastic approximation em algorithm using conditional particle filters, in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE (2013) 6274–6278. | DOI
[22] Lixoft SAS, Monolix (2020).
[23] , and , Asymptotic convergence rate of the algorithm for Gaussian mixtures. Neural Comput. 12 (2000) 2881–2907. | DOI
[24] et al., On the rate of convergence of the ECM algorithm. Ann. Stat. 22 (1994) 326–339. | MR | Zbl
[25] , and , Estimation in nonlinear mixed-effects models using heavy-tailed distributions. Stat. Comput. 22 (2012) 121–139. | MR | Zbl | DOI
[26] and , Extension of the SAEM algorithm for nonlinear mixed models with 2 levels of random effects. Biostatistics 10 (2009) 121–135. | Zbl | DOI
[27] and , Mixture densities, maximum likelihood and the algorithm. SIAM Rev. 26 (1984) 195–239. | MR | Zbl | DOI
[28] , and , Extension of the SAEM algorithm to left-censored data in nonlinear mixed-effects model: application to HIV dynamics model. Comput. Stat. Data Anal. 51 (2006) 1562–1574. | MR | Zbl | DOI
[29] , , and , Learning spatiotemporal trajectories from manifold-valued longitudinal data. Adv. Neural Inf. Process. Syst. (2015) 2404–2412.
[30] , , , , , , , , , , et al., Experimental treatment with Favipiravir for Ebola virus disease (the JIKI trial): a historically controlled, single-arm proof-of-concept trial in guinea. PLoS Med. 13 (2016) e1001967. | DOI
[31] , An analysis of the EM algorithm and entropy-like proximal point methods. Math Oper. Res. 29 (2004) 27–44. | MR | Zbl | DOI
[32] , Em algorithms for nonlinear mixed effects models. Comput. Stat. Data Anal. 51 (2007) 3244–3256. | MR | Zbl | DOI
[33] and , A Monte Carlo implementation of the algorithm and the poor man’s data augmentation algorithms. J. Am. Stat. Assoc. 85 (1990) 699–704. | DOI
[34] , On the convergence properties of the EM algorithm. Ann. Stat. (1983) 95–103. | MR | Zbl
Cité par Sources :
