Model selection for (auto-)regression with dependent data
ESAIM: Probability and Statistics, Tome 5 (2001), pp. 33-49.

In this paper, we study the problem of non parametric estimation of an unknown regression function from dependent data with sub-gaussian errors. As a particular case, we handle the autoregressive framework. For this purpose, we consider a collection of finite dimensional linear spaces (e.g. linear spaces spanned by wavelets or piecewise polynomials on a possibly irregular grid) and we estimate the regression function by a least-squares estimator built on a data driven selected linear space among the collection. This data driven choice is performed via the minimization of a penalized criterion akin to the Mallows’ ${C}_{p}$. We state non asymptotic risk bounds for our estimator in some ${Ł}_{2}$-norm and we show that it is adaptive in the minimax sense over a large class of Besov balls of the form ${ℬ}_{\alpha ,p,\infty }\left(R\right)$ with $p\ge 1$.

Classification : 62G08,  62J02
Mots clés : nonparametric regression, least-squares estimator, adaptive estimation, autoregression, mixing processes
@article{PS_2001__5__33_0,
author = {Baraud, Yannick and Comte, F. and Viennet, G.},
title = {Model selection for (auto-)regression with dependent data},
journal = {ESAIM: Probability and Statistics},
pages = {33--49},
publisher = {EDP-Sciences},
volume = {5},
year = {2001},
zbl = {0990.62035},
mrnumber = {1845321},
language = {en},
url = {http://www.numdam.org/item/PS_2001__5__33_0/}
}
TY  - JOUR
AU  - Baraud, Yannick
AU  - Comte, F.
AU  - Viennet, G.
TI  - Model selection for (auto-)regression with dependent data
JO  - ESAIM: Probability and Statistics
PY  - 2001
DA  - 2001///
SP  - 33
EP  - 49
VL  - 5
PB  - EDP-Sciences
UR  - http://www.numdam.org/item/PS_2001__5__33_0/
UR  - https://zbmath.org/?q=an%3A0990.62035
UR  - https://www.ams.org/mathscinet-getitem?mr=1845321
LA  - en
ID  - PS_2001__5__33_0
ER  - 
Baraud, Yannick; Comte, F.; Viennet, G. Model selection for (auto-)regression with dependent data. ESAIM: Probability and Statistics, Tome 5 (2001), pp. 33-49. http://www.numdam.org/item/PS_2001__5__33_0/

[1] H. Akaike, Information theory and an extension of the maximum likelihood principle, in Proc. 2nd International Symposium on Information Theory, edited by P.N. Petrov and F. Csaki. Akademia Kiado, Budapest (1973) 267-281. | MR 483125 | Zbl 0283.62006

[2] H. Akaike, A new look at the statistical model identification. IEEE Trans. Automat. Control 19 (1984) 716-723. | MR 423716 | Zbl 0314.62039

[3] P. Ango Nze, Geometric and subgeometric rates for markovian processes in the neighbourhood of linearity. C. R. Acad. Sci. Paris 326 (1998) 371-376. | MR 1648493 | Zbl 0918.60052

[4] Y. Baraud, Model selection for regression on a fixed design. Probab. Theory Related Fields 117 (2000) 467-493. | MR 1777129 | Zbl 0997.62027

[5] Y. Baraud, Model selection for regression on a random design, Preprint 01-10. DMA, École Normale Supérieure (2001). | Numdam | MR 1918295

[6] Y. Baraud, F. Comte and G. Viennet, Adaptive estimation in autoregression or $\beta$-mixing regression via model selection. Ann. Statist. (to appear). | MR 1865343 | Zbl 1012.62034

[7] A. Barron, L. Birgé and P. Massart, Risks bounds for model selection via penalization. Probab. Theory Related Fields 113 (1999) 301-413. | MR 1679028 | Zbl 0946.62036

[8] L. Birgé and P. Massart, An adaptive compression algorithm in Besov spaces. Constr. Approx. 16 (2000) 1-36. | MR 1848840 | Zbl 1004.41006

[9] L. Birgé and Y. Rozenholc, How many bins must be put in a regular histogram. Working paper (2001).

[10] A. Cohen, I. Daubechies and P. Vial, Wavelet and fast wavelet transform on an interval. Appl. Comput. Harmon. Anal. 1 (1993) 54-81. | MR 1256527 | Zbl 0795.42018

[11] I. Daubechies, Ten lectures on wavelets. SIAM: Philadelphia (1992). | MR 1162107 | Zbl 0776.42018

[12] R.A. Devore and C.G. Lorentz, Constructive Approximation. Springer-Verlag (1993). | MR 1261635 | Zbl 0797.41016

[13] D.L. Donoho and I.M. Johnstone, Minimax estimation via wavelet shrinkage. Ann. Statist. 26 (1998) 879-921. | MR 1635414 | Zbl 0935.62041

[14] P. Doukhan, Mixing properties and examples. Springer-Verlag (1994). | MR 1312160 | Zbl 0801.60027

[15] M. Duflo, Random Iterative Models. Springer, Berlin, New-York (1997). | MR 1485774 | Zbl 0868.62069

[16] M. Hoffmann, On nonparametric estimation in nonlinear AR(1)-models. Statist. Probab. Lett. 44 (1999) 29-45. | MR 1706307 | Zbl 0954.62049

[17] I.A. Ibragimov, On the spectrum of stationary Gaussian sequences satisfying the strong mixing condition I: Necessary conditions. Theory Probab. Appl. 10 (1965) 85-106. | MR 174091 | Zbl 0131.18101

[18] M. Kohler, On optimal rates of convergence for nonparametric regression with random design, Working Paper. Stuttgart University (1997).

[19] A.R. Kolmogorov and Y.A. Rozanov, On the strong mixing conditions for stationary Gaussian sequences. Theory Probab. Appl. 5 (1960) 204-207. | Zbl 0106.12005

[20] K.C. Li, Asymptotic optimality for ${C}_{p}$, ${C}_{l}$ cross-validation and generalized cross-validation: Discrete index set. Ann. Statist. 15 (1987) 958-975. | MR 902239 | Zbl 0653.62037

[21] G.G. Lorentz, M. Von Golitschek and Y. Makokov, Constructive Approximation, Advanced Problems. Springer, Berlin (1996). | MR 1393437 | Zbl 0910.41001

[22] C.L. Mallows, Some comments on ${C}_{p}$. Technometrics 15 (1973) 661-675. | Zbl 0269.62061

[23] A. Meyer, Quelques inégalités sur les martingales d'après Dubins et Freedman, Séminaire de Probabilités de l'Université de Strasbourg. Vols. 68/69 (1969) 162-169. | Numdam | Zbl 0211.21802

[24] D.S. Modha and E. Masry, Minimum complexity regression estimation with weakly dependent observations. IEEE Trans. Inform. Theory 42 (1996) 2133-2145. | MR 1447519 | Zbl 0868.62015

[25] D.S. Modha and E. Masry, Memory-universal prediction of stationary random processes. IEEE Trans. Inform. Theory 44 (1998) 117-133. | MR 1486652 | Zbl 0938.62106

[26] M. Neumann and J.-P. Kreiss, Regression-type inference in nonparametric autoregression. Ann. Statist. 26 (1998) 1570-1613. | MR 1647701 | Zbl 0935.62049

[27] B.T. Polyak and A. Tsybakov, A family of asymptotically optimal methods for choosing the order of a projective regression estimate. Theory Probab. Appl. 37 (1992) 471-481. | MR 1214355 | Zbl 0806.62031

[28] R. Shibata, Selection of the order of an autoregressive model by Akaike's information criterion. Biometrika 63 (1976) 117-126. | Zbl 0358.62048

[29] R. Shibata, An optimal selection of regression variables. Biometrika 68 (1981) 45-54. | MR 614940 | Zbl 0464.62054

[30] S. Van De Geer, Exponential inequalities for martingales, with application to maximum likelihood estimation for counting processes. Ann. Statist. 23 (1995) 1779-1801. | MR 1370307 | Zbl 0852.60019

[31] V.A. Volonskii and Y.A. Rozanov, Some limit theorems for random functions. I. Theory Probab. Appl. 4 (1959) 179-197. | MR 105741 | Zbl 0092.33502