Models are often defined through conditional rather than joint distributions, but it can be difficult to check whether the conditional distributions are compatible, i.e. whether there exists a joint probability distribution which generates them. When they are compatible, a Gibbs sampler can be used to sample from this joint distribution. When they are not, the Gibbs sampling algorithm may still be applied, resulting in a “pseudo-Gibbs sampler”. We show its stationary probability distribution to be the optimal compromise between the conditional distributions, in the sense that it minimizes a mean squared misfit between them and its own conditional distributions. This allows us to perform Objective Bayesian analysis of correlation parameters in Kriging models by using univariate conditional Jeffreys-rule posterior distributions instead of the widely used multivariate Jeffreys-rule posterior. This strategy makes the full-Bayesian procedure tractable. Numerical examples show it has near-optimal frequentist performance in terms of prediction interval coverage.
Accepté le :
DOI : 10.1051/ps/2018023
Keywords: Incompatibility, conditional distribution, Markov kernel, optimal compromise, Kriging, reference prior, integrated likelihood, Gibbs sampling, posterior propriety, frequentist coverage
Muré, Joseph 1
@article{PS_2019__23__271_0,
author = {Mur\'e, Joseph},
title = {Optimal compromise between incompatible conditional probability distributions, with application to {Objective} {Bayesian} {Kriging}},
journal = {ESAIM: Probability and Statistics},
pages = {271--309},
year = {2019},
publisher = {EDP Sciences},
volume = {23},
doi = {10.1051/ps/2018023},
mrnumber = {3963529},
zbl = {1420.62117},
language = {en},
url = {https://www.numdam.org/articles/10.1051/ps/2018023/}
}
TY - JOUR AU - Muré, Joseph TI - Optimal compromise between incompatible conditional probability distributions, with application to Objective Bayesian Kriging JO - ESAIM: Probability and Statistics PY - 2019 SP - 271 EP - 309 VL - 23 PB - EDP Sciences UR - https://www.numdam.org/articles/10.1051/ps/2018023/ DO - 10.1051/ps/2018023 LA - en ID - PS_2019__23__271_0 ER -
%0 Journal Article %A Muré, Joseph %T Optimal compromise between incompatible conditional probability distributions, with application to Objective Bayesian Kriging %J ESAIM: Probability and Statistics %D 2019 %P 271-309 %V 23 %I EDP Sciences %U https://www.numdam.org/articles/10.1051/ps/2018023/ %R 10.1051/ps/2018023 %G en %F PS_2019__23__271_0
Muré, Joseph. Optimal compromise between incompatible conditional probability distributions, with application to Objective Bayesian Kriging. ESAIM: Probability and Statistics, Tome 23 (2019), pp. 271-309. doi: 10.1051/ps/2018023
[1] and , Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Vol. 55 of Applied Mathematics Series. National Bureau of Standards (1964). | MR | Zbl
[2] , On the consistent separation of scale and variance for Gaussian random fields. Ann. Stat. 38 (2010) 870–893. | MR | Zbl | DOI
[3] , and , Conditionally specified distributions: an introduction. Stat. Sci. 16 (2001) 268–269. | MR | Zbl | DOI
[4] , The case for objective bayesian analysis. Bayesian Anal. 1 (2006) 385–402. | MR | Zbl | DOI
[5] and , On the development of reference priors. Bayesian Stat. 4 (1992) 35–60. | MR | DOI
[6] , and , Objective Bayesian analysis of spatially correlated data. J. Am. Stat. Assoc. 96 (2001) 1361–1374. | MR | Zbl | DOI
[7] , and . Overall objective priors. Bayesian Anal. 10 (2015) 189–221. | MR | Zbl | DOI
[8] , Reference analysis, in Vol. 25 of Handbook of Statistics, edited by and . Elsevier (2005) 17–90. | MR | DOI
[9] , and , Sélection bayésienne de variables en régression linéaire. J. Soc. Fr. Statistique 147 (2005) 59–79. | MR | Zbl
[10] and , Jeffreys’ prior is asymptotically least favorable under entropy risk. J. Stat. Plan. Inference 41 (1994) 37–60. | MR | Zbl | DOI
[11] and , Compatible prior distributions. Bayesian Methods with Applications to Science, Policy and Official Statistics, Selected Papers from ISBA 2000: The Sixth World Meeting of the International Society for Bayesian Analysis, edited by . Eurostat, Luxembourg (2001) 109–118.
[12] and , Comment on “Conditionally specified distributions: an introduction” by B.C. Arnold, E. Castillo and J.M. Sarabia. Stat. Sci. 16 (2001) 268–269.
[13] , and , Robust Gaussian stochastic process emulation. Ann. Stat. 46 (2018) 3038–3066. | MR | Zbl
[14] , , and , Scan order in Gibbs sampling: Models in which it matters and bounds on how much. Adv.Neural Inf. Process. Syst. 29 (2016) 1–9.
[15] , , , and , Dependency networks for inference, collaborative filtering, and data visualization. J. Mach. Learn. Res. 1 (2000) 49–75. | Zbl
[16] and , Functional compatibility, Markov chains, and Gibbs sampling with improper posteriors. J. Comput. Graph. Stat. 7 (1998) 42–60. | MR
[17] , New light on the correlation coefficient and its transforms. J. R. Stat. Soc. Ser. B (Methodol.) 15 (1953) 193–232. | MR | Zbl
[18] and , Mining Geostatistics. Academic Press, New York (1978).
[19] and , Objective bayesian analysis of spatial data with uncertain nugget and range parameters. Can. J. Stat. 40 (2012) 304–327. | MR | Zbl | DOI
[20] and Bayesian calibration of computer models. J. R. Stat. Soc., Ser. B (Stat. Methodol.) 63 (2001) 425–464. | MR | Zbl | DOI
[21] and , Pseudo-Gibbs sampler for discrete conditional distributions. Ann. Inst. Stat. Math. (2017) 1–13. | MR
[22] , and , Exactly and almost compatible joint distributions for high-dimensional discrete conditional distributions. J. Multivar. Anal. 157 (2017) 115–123. | MR | Zbl | DOI
[23] and , Analysis of computer experiments using penalized likelihood in Gaussian Kriging models. Technometrics 47 (2005) 111–120. | MR | DOI
[24] and , Improving Gibbs Sampler Scan Quality with DoGS, in Vol. 70 of Proceedings of the 34th International Conference on Machine Learning. (2017) 2469–2477.
[25] , Default priors for Gaussian processes. Ann. Stat. 33 (2005) 556–582. | MR | Zbl | DOI
[26] and , Gaussian Processes for Machine Learning. MIT Press (2006). | MR | Zbl
[27] , and , Objective bayesian analysis for a spatial model with nugget effects. J. Stat. Plan. Inference 142 (2012) 1933–1946. | MR | Zbl | DOI
[28] , and , Objective bayesian analysis of spatial models with separable correlation functions. Can. J. Stat. 41 (2013) 488–507. | MR | Zbl | DOI
[29] , The Bayesian Choice : From Decision-Theoretic Foundations to Computational Implementation. Springer-Verlag, New York (2007). | MR
[30] and , Compatible prior distributions for directed acyclic graph models. J. R. Stat. Soc., Ser. B (Stat. Methodol.) 66 (2004) 47–61. | MR | Zbl | DOI
[31] , and , The Design and Analysis of Computer Experiments. Springer-Verlag, New York (2003). | MR | Zbl | DOI
[32] , Interpolation of Spatial Data. Some Theory for Kriging. Springer Series in Statistics. Springer-Verlag, New York (1999). | MR | Zbl | DOI
[33] and , A Catalog of Noninformative Priors. Institute of Statistics and Decision Sciences, Duke University (1996).
[34] and , Compatible and incompatible abstractions in Bayesian networks. Knowl.-Based Syst. 62 (2014) 84–97. | DOI
[35] , Inconsistent Estimation and Asymptotically Equal Interpolations in Model-based Geostatistics. J. Am. Stat. Assoc. 99 (2004) 250–261. | MR | Zbl | DOI
Cité par Sources :
