In this paper, we study the problem of non parametric estimation of an unknown regression function from dependent data with sub-gaussian errors. As a particular case, we handle the autoregressive framework. For this purpose, we consider a collection of finite dimensional linear spaces (e.g. linear spaces spanned by wavelets or piecewise polynomials on a possibly irregular grid) and we estimate the regression function by a least-squares estimator built on a data driven selected linear space among the collection. This data driven choice is performed via the minimization of a penalized criterion akin to the Mallows’ . We state non asymptotic risk bounds for our estimator in some -norm and we show that it is adaptive in the minimax sense over a large class of Besov balls of the form with .
Keywords: nonparametric regression, least-squares estimator, adaptive estimation, autoregression, mixing processes
@article{PS_2001__5__33_0, author = {Baraud, Yannick and Comte, F. and Viennet, G.}, title = {Model selection for (auto-)regression with dependent data}, journal = {ESAIM: Probability and Statistics}, pages = {33--49}, publisher = {EDP-Sciences}, volume = {5}, year = {2001}, mrnumber = {1845321}, zbl = {0990.62035}, language = {en}, url = {http://www.numdam.org/item/PS_2001__5__33_0/} }
TY - JOUR AU - Baraud, Yannick AU - Comte, F. AU - Viennet, G. TI - Model selection for (auto-)regression with dependent data JO - ESAIM: Probability and Statistics PY - 2001 SP - 33 EP - 49 VL - 5 PB - EDP-Sciences UR - http://www.numdam.org/item/PS_2001__5__33_0/ LA - en ID - PS_2001__5__33_0 ER -
Baraud, Yannick; Comte, F.; Viennet, G. Model selection for (auto-)regression with dependent data. ESAIM: Probability and Statistics, Volume 5 (2001), pp. 33-49. http://www.numdam.org/item/PS_2001__5__33_0/
[1] Information theory and an extension of the maximum likelihood principle, in Proc. 2nd International Symposium on Information Theory, edited by P.N. Petrov and F. Csaki. Akademia Kiado, Budapest (1973) 267-281. | MR | Zbl
,[2] A new look at the statistical model identification. IEEE Trans. Automat. Control 19 (1984) 716-723. | MR | Zbl
,[3] Geometric and subgeometric rates for markovian processes in the neighbourhood of linearity. C. R. Acad. Sci. Paris 326 (1998) 371-376. | MR | Zbl
,[4] Model selection for regression on a fixed design. Probab. Theory Related Fields 117 (2000) 467-493. | MR | Zbl
,[5] Model selection for regression on a random design, Preprint 01-10. DMA, École Normale Supérieure (2001). | Numdam | MR
,[6] Adaptive estimation in autoregression or -mixing regression via model selection. Ann. Statist. (to appear). | MR | Zbl
, and ,[7] Risks bounds for model selection via penalization. Probab. Theory Related Fields 113 (1999) 301-413. | MR | Zbl
, and ,[8] An adaptive compression algorithm in Besov spaces. Constr. Approx. 16 (2000) 1-36. | MR | Zbl
and ,[9] How many bins must be put in a regular histogram. Working paper (2001).
and ,[10] Wavelet and fast wavelet transform on an interval. Appl. Comput. Harmon. Anal. 1 (1993) 54-81. | MR | Zbl
, and ,[11] Ten lectures on wavelets. SIAM: Philadelphia (1992). | MR | Zbl
,[12] Constructive Approximation. Springer-Verlag (1993). | MR | Zbl
and ,[13] Minimax estimation via wavelet shrinkage. Ann. Statist. 26 (1998) 879-921. | MR | Zbl
and ,[14] Mixing properties and examples. Springer-Verlag (1994). | MR | Zbl
,[15] Random Iterative Models. Springer, Berlin, New-York (1997). | MR | Zbl
,[16] On nonparametric estimation in nonlinear AR(1)-models. Statist. Probab. Lett. 44 (1999) 29-45. | MR | Zbl
,[17] On the spectrum of stationary Gaussian sequences satisfying the strong mixing condition I: Necessary conditions. Theory Probab. Appl. 10 (1965) 85-106. | MR | Zbl
,[18] On optimal rates of convergence for nonparametric regression with random design, Working Paper. Stuttgart University (1997).
,[19] On the strong mixing conditions for stationary Gaussian sequences. Theory Probab. Appl. 5 (1960) 204-207. | Zbl
and ,[20] Asymptotic optimality for , cross-validation and generalized cross-validation: Discrete index set. Ann. Statist. 15 (1987) 958-975. | MR | Zbl
,[21] Constructive Approximation, Advanced Problems. Springer, Berlin (1996). | MR | Zbl
, and ,[22] Some comments on . Technometrics 15 (1973) 661-675. | Zbl
,[23] Quelques inégalités sur les martingales d'après Dubins et Freedman, Séminaire de Probabilités de l'Université de Strasbourg. Vols. 68/69 (1969) 162-169. | Numdam | Zbl
,[24] Minimum complexity regression estimation with weakly dependent observations. IEEE Trans. Inform. Theory 42 (1996) 2133-2145. | MR | Zbl
and ,[25] Memory-universal prediction of stationary random processes. IEEE Trans. Inform. Theory 44 (1998) 117-133. | MR | Zbl
and ,[26] Regression-type inference in nonparametric autoregression. Ann. Statist. 26 (1998) 1570-1613. | MR | Zbl
and ,[27] A family of asymptotically optimal methods for choosing the order of a projective regression estimate. Theory Probab. Appl. 37 (1992) 471-481. | MR | Zbl
and ,[28] Selection of the order of an autoregressive model by Akaike's information criterion. Biometrika 63 (1976) 117-126. | Zbl
,[29] An optimal selection of regression variables. Biometrika 68 (1981) 45-54. | MR | Zbl
,[30] Exponential inequalities for martingales, with application to maximum likelihood estimation for counting processes. Ann. Statist. 23 (1995) 1779-1801. | MR | Zbl
,[31] Some limit theorems for random functions. I. Theory Probab. Appl. 4 (1959) 179-197. | MR | Zbl
and ,