This paper is devoted to the estimation of a partial graphical model with a structural Bayesian penalization. Precisely, we are interested in the linear regression setting where the estimation is made through the direct links between potentially high-dimensional predictors and multiple responses, since it is known that Gaussian graphical models enable to exhibit direct links only, whereas coefficients in linear regressions contain both direct and indirect relations (due e.g. to strong correlations among the variables). A smooth penalty reflecting a generalized Gaussian Bayesian prior on the covariates is added, either enforcing patterns (like row structures) in the direct links or regulating the joint influence of predictors. We give a theoretical guarantee for our method, taking the form of an upper bound on the estimation error arising with high probability, provided that the model is suitably regularized. Empirical studies on synthetic data and a real dataset are conducted.
Keywords: High-dimensional linear regression, partial graphical model, structural penalization, sparsity, convex optimization
@article{PS_2021__25_1_298_0,
author = {Okome Obiang, Eunice and J\'ez\'equel, Pascal and Pro{\"\i}a, Fr\'ed\'eric},
title = {A partial graphical model with a structural prior on the direct links between predictors and responses},
journal = {ESAIM: Probability and Statistics},
pages = {298--324},
year = {2021},
publisher = {EDP-Sciences},
volume = {25},
doi = {10.1051/ps/2021010},
mrnumber = {4283606},
zbl = {1469.62282},
language = {en},
url = {https://www.numdam.org/articles/10.1051/ps/2021010/}
}
TY - JOUR AU - Okome Obiang, Eunice AU - Jézéquel, Pascal AU - Proïa, Frédéric TI - A partial graphical model with a structural prior on the direct links between predictors and responses JO - ESAIM: Probability and Statistics PY - 2021 SP - 298 EP - 324 VL - 25 PB - EDP-Sciences UR - https://www.numdam.org/articles/10.1051/ps/2021010/ DO - 10.1051/ps/2021010 LA - en ID - PS_2021__25_1_298_0 ER -
%0 Journal Article %A Okome Obiang, Eunice %A Jézéquel, Pascal %A Proïa, Frédéric %T A partial graphical model with a structural prior on the direct links between predictors and responses %J ESAIM: Probability and Statistics %D 2021 %P 298-324 %V 25 %I EDP-Sciences %U https://www.numdam.org/articles/10.1051/ps/2021010/ %R 10.1051/ps/2021010 %G en %F PS_2021__25_1_298_0
Okome Obiang, Eunice; Jézéquel, Pascal; Proïa, Frédéric. A partial graphical model with a structural prior on the direct links between predictors and responses. ESAIM: Probability and Statistics, Tome 25 (2021), pp. 298-324. doi: 10.1051/ps/2021010
[1] and , Scalable training of L1-regularized log-linear models. Proc. 24th Inte. Conf. Mach. Learning (2007) 33–40.
[2] , and , Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Res. 9 (2008) 485–516. | MR | Zbl
[3] , , and , A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28 (2008) 253–263. | MR | Zbl | DOI
[4] and , Convex Optimization. Cambridge University Press (2004). | MR | Zbl | DOI
[5] , , and , Covariate-adjusted precision matrix estimation with an application in genetical genomics. Biometrika 100 (2013) 139–156. | MR | Zbl | DOI
[6] , and , A constrained minimization approach to sparse precision matrix estimation. J. Am. Stat. Assoc. 106 (2011) 594–607. | MR | Zbl | DOI
[7] , and , Structured regularization for conditional Gaussian graphical models. Stat. Comput. 27 (2017) 789–804. | MR | Zbl | DOI
[8] , and , Network exploration via the adaptive Lasso and SCAD penalties. Ann. Appl. Stat. 3 (2009) 521–541. | MR | Zbl
[9] , and , Sparse inverse covariance estimation with the graphical Lasso. Biostatistics 9 (2008) 432–441. | Zbl | DOI
[10] , Introduction to High-Dimensional Statistics. Chapman & Hall/CRC Monographs on Statistics & Applied Probability. Taylor & Francis (2014). | Zbl
[11] , and , Statistical Learning with Sparsity: The Lasso and Generalizations. Chapman & Hall/CRC Monographs on Statistics and Applied Probability. CRC Press (2015). | MR | Zbl
[12] and , Matrix Analysis (Second Edition). Cambridge University Press, Cambridge, New York (2012). | MR | Zbl | DOI
[13] , and , High-dimensional sparse inverse covariance estimation using greedy methods. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, volume 22 of Proceedings of Machine Learning Research. PMLR (2012) 574–582.
[14] and , Simultaneous multiple response regression and inverse covariance matrix estimation via penalized Gaussian maximum likelihood. J. Multivariate. Anal. 111 (2012) 241–255. | MR | Zbl | DOI
[15] , Smooth optimization approach for sparse covariance selection. Siam. J. Optimiz. 19 (2009) 1807–1827. | MR | Zbl | DOI
[16] , , and , Handbook of Graphical Models. Chapman & Hall/CRC Handbooks of Modern Statistical Methods. CRC Press (2018). | MR | Zbl
[17] and , High-dimensional graphs and variable selection with the Lasso. Ann. Stat. 34 (2006) 1436–1462. | MR | Zbl | DOI
[18] , , and , Parameter estimation for multivariate generalized Gaussian distributions. IEEE Trans. Signal. Process. 61 (2013) 5960–5971. | MR | Zbl | DOI
[19] , , and , Partial correlation estimation by joint sparse regression models. J. Am. Stat. Assoc. 104 (2009) 735–746. | MR | Zbl | DOI
[20] and , Functional Data Analysis, 2nd ed. Springer, New York (2006). | Zbl | MR
[21] , , and , High-dimensional covariance estimation by minimizing -penalized log-determinant divergence. Electr. J. Stat. 5 (2011) 935–980. | MR | Zbl
[22] , and , Bayesian Statistics and Marketing. Wiley Series in Probability and Statistics. Wiley (2012). | MR | Zbl
[23] , and , Sparse multivariate regression with covariance estimation. J. Comput. Graph. Stat. 19 (2010) 947–962. | MR | DOI
[24] , The structured elastic net for quantile regression and support vector classification. Stat. Comput. 22 (2012) 153–168. | MR | Zbl | DOI
[25] , and , Feature selection guided by structural information. Ann. Appl. Stat. 4 (2010) 1056–1080. | MR | Zbl | DOI
[26] and , Joint estimation of structured sparsity and output structure in multiple-output regression via inverse-covariance regularization. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics. Vol. 22 of Proceedings of Machine Learning Research. PMLR (2012) 1081–1089.
[27] and , A sparse conditional Gaussian graphical model for analysis of genetical genomics data. Ann. Appl. Stat. 5 (2011) 2630–2650. | MR | Zbl
[28] and , Model selection and estimation in the Gaussian graphical model. Biometrika 94 (2007) 19–35. | MR | Zbl | DOI
[29] and , Partial Gaussian graphical model estimation. IEEE Trans. Inf. Theory 60 (2014) 1673–1687. | MR | Zbl | DOI
Cité par Sources :
