Spectral analysis of the Gram matrix of mixture models

Benaych-Georges, Florent; Couillet, Romain

ESAIM: Probability and Statistics, Tome 20 (2016), pp. 217-237.

This text is devoted to the asymptotic study of some spectral properties of the Gram matrix $W^{T}W$ built upon a collection $w_1,...,w_n\in R^p$ of random vectors (the columns of $W$), as both the number $n$ of observations and the dimension $p$ of the observations tend to infinity and are of similar order of magnitude. The random vectors $w_1,...,w_n$ are independent observations, each of them belonging to one of $k$ classes ${𝒞}_1,...,{𝒞}_k$. The observations of each class ${𝒞}_a$ ($1\le a\le k$) are characterized by their distribution $𝒩(0,p^{-1}{C}_a)$, where $C_1,...,C_k$ are some non negative definite $p\times p$ matrices. The cardinality $n_a$ of class ${𝒞}_a$ and the dimension $p$ of the observations are such that $n_a/n$ ($1\le a\le k$) and $p/n$ stay bounded away from $0$ and $+\infty$. We provide deterministic equivalents to the empirical spectral distribution of $W^{T}W$ and to the matrix entries of its resolvent (as well as of the resolvent of $W{W}^T$). These deterministic equivalents are defined thanks to the solutions of a fixed-point system. Besides, we prove that $W^{T}W$ has asymptotically no eigenvalues outside the bulk of its spectrum, defined thanks to these deterministic equivalents. These results are directly used in our companion paper [R. Couillet and F. Benaych-Georges, Electron. J. Stat. 10 (2016) 1393–1454.], which is devoted to the analysis of the spectral clustering algorithm in large dimensions. They also find applications in various other fields such as wireless communications where functionals of the aforementioned resolvents allow one to assess the communication performance across multi-user multi-antenna channels.

Classification : 60B20, 15B52, 62H30
Mots clés : Random matrices, extreme eigenvalue statistics, mixture models, spectral clustering

O. Ajanki, L. Erdös and T. Kruger Quadratic vector equations on complex upper half-plane (2015).

G. Anderson, A. Guionnet and O. Zeitouni, An Introduction to Random Matrices. Vol. 118 of Cambridge Studies Advanced Math. (2009). | MR | Zbl

Z.D. Bai and J.W. Silverstein, No eigenvalues outside the support of the limiting spectral distribution of large dimensional sample covariance matrices. Ann. Probab. (1998) 26 316–345. | MR | Zbl

J. Baik, G. Ben Arous and S. Péché, Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Ann. Probab. 33 (2005) 1643–1697. | DOI | MR | Zbl

F. Benaych-Georges and R.N. Rao, The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices. Adv. Math. (2011) 227 494–521. | DOI | MR | Zbl

F. Benaych-Georges and R.N. Rao, The singular values and vectors of low rank perturbations of large rectangular random matrices. J. Multivariate Anal. 111 (2012) 120–135. | DOI | MR | Zbl

M. Capitaine, Additive/multiplicative free subordination property and limiting eigenvectors of spiked additive deformations of Wigner matrices and spiked sample covariance matrices. J. Theor. Probab. 26 (2013) 595–648. | DOI | MR | Zbl

F. Chapon, R. Couillet, W. Hachem and X. Mestre, The outliers among the singular values of large rectangular random matrices with additive fixed rank deformation. Markov Process. Relat. Fields 20 (2014) 183–228. | MR | Zbl

R. Couillet and W. Hachem, Analysis of the limiting spectral measure of large random matrices of the separable covariance type. Random Matrices: Theory Appl. 3 (2014) 1450016. | DOI | MR | Zbl

R. Couillet and F. Benaych-Georges, Kernel spectral clustering of large dimensional data. Electron. J. Stat. 10 (2016) 1393–1454. | DOI | MR | Zbl

R. Couillet, M. Debbah and J.W. Silverstein, A deterministic equivalent for the analysis of correlated MIMO multiple access channels. IEEE Trans. Inform. Theory 57 (2011) 3493–3514. | DOI | MR | Zbl

L. Erdös, B. Schlein and H.-T. Yau, Semicircle law on short scales and delocalization of eigenvectors for Wigner random matrices. Ann. Prob. 37 (2009). | DOI | MR | Zbl

T. Hastie, R. Tibshirani and J. Friedman, The elements of statistical learning. Data mining, inference, and prediction. Springer Series in Statistics, 3nd edition. Springer, New York (2009). | MR | Zbl

R.A. Horn and C.R. Johnson, Matrix Analysis. Cambridge University Press (2013). | MR | Zbl

R.A. Horn and C.R. Johnson, Topics in Matrix Analysis. Cambridge University Press (1991). | MR

G. James, D. Witten, T. Hastie and R. Tibshirani, An introduction to statistical learning. With applications in R. Vol. 103 of Springer Texts in Statistics. Springer, New York (2013). | MR | Zbl

I.M. Johnstone, On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 (2001) 295-327. | DOI | MR | Zbl

A. Kammoun, M. Kharouf, W. Hachem and J. Najim, A central limit theorem for the SINR at the LMMSE estimator output for large-dimensional signals. IEEE Trans. Inform. Theory 55 (2009) 5048–5063. | DOI | MR | Zbl

R. Kannan and S. Vempala, Spectral algorithms. Found. Trends Theoret. Comput. Sci. 4 (2009) 157–288. | DOI | MR | Zbl

V. Kargin, A concentration inequality and a local law for the sum of two random matrices. Probab. Theory Related Fields 154 (2012) 677–702. | DOI | MR | Zbl

P. Loubaton and P. Vallet, Almost sure localization of the eigenvalues in a Gaussian information plus noise model. Applications to the spiked models. Electron. J. Probab. 16 (2011) 1934–1959. | DOI | MR | Zbl

U. Von Luxburg, A tutorial on spectral clustering. Stat. Comput. 17 (2007) 395–416. | DOI | MR

V.A. Marcenko and L.A. Pastur, Distribution of eigenvalues for some sets of random matrices. Sb. Math. 1 (1967) 457–483. | DOI | Zbl

J.W. Silverstein and S. Choi, Analysis of the limiting spectral distribution of large dimensional random matrices. J. Multivariate Anal. 54 (1995) 295–309. | DOI | MR | Zbl