Bayesian calibration of a numerical code for prediction
[Theory of code calibration and application to the prediction of a photovoltaic power plant electricity production]
Journal de la société française de statistique, Tome 160 (2019) no. 1, pp. 1-30.

Field experiments are often difficult and expensive to carry out. To bypass these issues, industrial companies have developed computational codes. These codes are intended to be representative of the physical system, but come with a certain number of problems. Despite continuous code development, the difference between the code outputs and experiments can remain significant. Two kinds of uncertainties are observed. The first one comes from the difference between the physical phenomenon and the values recorded experimentally. The second concerns the gap between the code and the physical system. To reduce this difference, often named model bias, discrepancy, or model error, computer codes are generally complexified in order to make them more realistic. These improvements increase the computational cost of the code. Moreover, a code often depends on user-defined parameters in order to match field data as closely as possible. This estimation task is called calibration. This paper proposes a review of Bayesian calibration methods and is based on an application case which makes it possible to discuss the various methodological choices and to illustrate their divergences. This example is based on a code used to predict the power of a photovoltaic plant.

Les difficultés de mise en œuvre d’expériences de terrain ou de laboratoire, ainsi que les coûts associés, conduisent les sociétés industrielles à se tourner vers des codes numériques de calcul. Ces codes, censés être représentatifs des phénomènes physiques en jeu, entraînent néanmoins tout un cortège de problèmes. Le premier de ces problèmes provient de la volonté de prédire la réalité à partir d’un modèle informatique. En effet, le code doit être représentatif du phénomène et, par conséquent, être capable de simuler des données proches de la réalité. Or, malgré le constant développement du réalisme de ces codes, des erreurs de prédiction subsistent. Elles sont de deux natures différentes. La première provient de la différence entre le phénomène physique et les valeurs relevées expérimentalement. La deuxième concerne l’écart entre le code développé et le phénomène physique. Pour diminuer cet écart, souvent qualifié de biais ou d’erreur de modèle, les développeurs complexifient en général les codes, les rendant très chronophages dans certains cas. De plus, le code dépend de paramètres à fixer par l’utilisateur qui doivent être choisis pour correspondre au mieux aux données de terrain. L’estimation de ces paramètres propres au code s’appelle le calage. Ce papier propose une revue des méthodes de calage bayésien et s’appuie sur un cas d’application qui permet de discuter les divers choix méthodologiques et d’illustrer leurs divergences. Cet exemple s’appuie sur un code de calcul servant à prédire la puissance d’une centrale photovoltaïque.

Mots clés : Photovoltaic power plant, Bayesian calibration, Uncertainty quantification, Numerical code
@article{JSFS_2019__160_1_1_0,
     author = {Carmassi, Mathieu and Barbillon, Pierre and Chiodetti, Matthieu and Keller, Merlin and Parent, Eric},
     title = {Bayesian calibration of a numerical code for prediction},
     journal = {Journal de la soci\'et\'e fran\c{c}aise de statistique},
     pages = {1--30},
     publisher = {Soci\'et\'e fran\c{c}aise de statistique},
     volume = {160},
     number = {1},
     year = {2019},
     zbl = {07056837},
     language = {en},
     url = {http://www.numdam.org/item/JSFS_2019__160_1_1_0/}
}
TY  - JOUR
AU  - Carmassi, Mathieu
AU  - Barbillon, Pierre
AU  - Chiodetti, Matthieu
AU  - Keller, Merlin
AU  - Parent, Eric
TI  - Bayesian calibration of a numerical code for prediction
JO  - Journal de la société française de statistique
PY  - 2019
DA  - 2019///
SP  - 1
EP  - 30
VL  - 160
IS  - 1
PB  - Société française de statistique
UR  - http://www.numdam.org/item/JSFS_2019__160_1_1_0/
UR  - https://zbmath.org/?q=an%3A07056837
LA  - en
ID  - JSFS_2019__160_1_1_0
ER  - 
Carmassi, Mathieu; Barbillon, Pierre; Chiodetti, Matthieu; Keller, Merlin; Parent, Eric. Bayesian calibration of a numerical code for prediction. Journal de la société française de statistique, Tome 160 (2019) no. 1, pp. 1-30. http://www.numdam.org/item/JSFS_2019__160_1_1_0/

[1] Albert, Isabelle; Donnet, Sophie; Guihenneuc-Jouyaux, Chantal; Low-Choy, Samantha; Mengersen, Kerrie; Rousseau, Judith Combining expert opinions in prior elicitation, Bayesian Analysis, Volume 7 (2012) no. 3, pp. 503-532 | Zbl 1330.62105

[2] Bachoc, François; Bois, Guillaume; Garnier, Josselin; Martinez, Jean-Marc Calibration and improved prediction of computer models by universal Kriging, Nuclear Science and Engineering, Volume 176 (2014) no. 1, pp. 81-97

[3] Bayarri, Maria J; Berger, James O; Paulo, Rui; Sacks, Jerry; Cafeo, John A; Cavendish, James; Lin, Chin-Hsu; Tu, Jian A framework for validation of computer models, Technometrics, Volume 49 (2007) no. 2, pp. 138-154

[4] Brynjarsdóttir, J.; O’Hagan, Anthony Learning about physical parameters: The importance of model discrepancy, Inverse Problems, Volume 30 (2014) no. 11 | Article | Zbl 1307.60042

[5] Carmassi, Mathieu CaliCo: Code Calibration in a Bayesian Framework (2018) https://CRAN.R-project.org/package=CaliCo (R package version 0.1.0)

[6] Craig, Peter S; Goldstein, Michael; Rougier, Jonathan C; Seheult, Allan H Bayesian forecasting for complex systems using computer simulators, Journal of the American Statistical Association, Volume 96 (2001) no. 454, pp. 717-729 | Zbl 1017.62019

[7] Currin, Carla; Mitchell, Toby; Morris, Max; Ylvisaker, Don Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments, Journal of the American Statistical Association, Volume 86 (1991) no. 416, pp. 953-963

[8] Cox, Dennis D; Park, Jeong-Soo; Singer, Clifford E A statistical method for tuning a computer code to a data base, Computational statistics & data analysis, Volume 37 (2001) no. 1, pp. 77-92 | Zbl 1077.62547

[9] Damblin, G Contributions statistiques au calage et à la validation des codes de calcul (2015) (Ph. D. Thesis)

[10] Duffie, John A; Beckman, William A Solar engineering of thermal processes, John Wiley & Sons, 2013

[11] Damblin, Guillaume; Barbillon, Pierre; Keller, Merlin; Pasanisi, Alberto; Parent, Éric Adaptive numerical designs for the calibration of computer codes, SIAM/ASA Journal on Uncertainty Quantification, Volume 6 (2018) no. 1, pp. 151-179 | Zbl 1386.65058

[12] Damblin, Guillaume; Keller, Merlin; Barbillon, Pierre; Pasanisi, Alberto; Parent, Éric Bayesian model selection for the validation of computer codes, Quality and Reliability Engineering International, Volume 32 (2016) no. 6, pp. 2043-2054

[13] Da Veiga, Sébastien; Marrel, Amandine Gaussian process modeling with inequality constraints, Annales de la Faculté des Sciences de Toulouse, Volume 21 (2012) no. 3, pp. 529-555 | Numdam | Zbl 1279.60047

[14] Fang, Kai-Tai; Li, Runze; Sudjianto, Agus Design and modeling for computer experiments, CRC Press, 2005 | Zbl 1093.62117

[15] Gu, Mengyang; Wang, Long An improved approach to Bayesian computer model calibration and prediction, arXiv preprint arXiv:1707.08215 (2017)

[16] Hastings, W Keith Monte Carlo sampling methods using Markov chains and their applications, Biometrika, Volume 57 (1970) no. 1, pp. 97-109 | Zbl 0219.65008

[17] Higdon, Dave; Gattiker, James; Williams, Brian; Rightley, Maria Computer model calibration using high-dimensional output, Journal of the American Statistical Association, Volume 103 (2008) no. 482, pp. 570-583 | Zbl 05564511

[18] Higdon, Dave; Kennedy, Marc; Cavendish, James C; Cafeo, John A; Ryne, Robert D Combining field data and computer simulations for calibration and prediction, SIAM Journal on Scientific Computing, Volume 26 (2004) no. 2, pp. 448-466 | Zbl 1072.62018

[19] Jones, Donald R; Schonlau, Matthias; Welch, William J Efficient global optimization of expensive black-box functions, Journal of Global optimization, Volume 13 (1998) no. 4, pp. 455-492 | Zbl 0917.90270

[20] Kennedy, Marc C; O’Hagan, Anthony Bayesian calibration of computer models, Journal of the Royal Statistical Society: Series B (Statistical Methodology), Volume 63 (2001) no. 3, pp. 425-464 | Zbl 1007.62021

[21] Kennedy, M; O’Hagan, A Supplementary details on bayesian calibration of computer. Rap. tech., University of Nottingham, Statistics Section (2001)

[22] Liu, Fei; Bayarri, MJ; Berger, JO Modularization in Bayesian analysis, with emphasis on analysis of computer models, Bayesian Analysis, Volume 4 (2009) no. 1, pp. 119-150 | Zbl 1330.65033

[23] Luque, Antonio; Hegedus, Steven Handbook of photovoltaic science and engineering, John Wiley & Sons, 2011

[24] Morris, Max D; Mitchell, Toby J Exploratory designs for computational experiments, Journal of statistical planning and inference, Volume 43 (1995) no. 3, pp. 381-402 | Zbl 0813.62065

[25] Morris, Max D Factorial sampling plans for preliminary computational experiments, Technometrics, Volume 33 (1991) no. 2, pp. 161-174

[26] Martin, N; Ruiz, JM Calculation of the PV modules angular losses under field conditions by means of an analytical model, Solar Energy Materials and Solar Cells, Volume 70 (2001) no. 1, pp. 25-38

[27] Metropolis, Nicholas; Rosenbluth, Arianna W; Rosenbluth, Marshall N; Teller, Augusta H; Teller, Edward Equation of state calculations by fast computing machines, The journal of chemical physics, Volume 21 (1953) no. 6, pp. 1087-1092 | Zbl 1431.65006

[28] Oberkampf, William L; Sindir, M; Conlisk, AT Guide for the verification and validation of computational fluid dynamics simulations, American Institute of Aeronautics and Astronautics, Reston, VA (1998) | Zbl 0979.76001

[29] Plumlee, Matthew Bayesian Calibration of Inexact Computer Models, Journal of the American Statistical Association, Volume 112 (2017) no. 519, pp. 1274-1285 | Article

[30] Pronzato, Luc; Müller, Werner G Design of computer experiments: space filling and beyond, Statistics and Computing, Volume 22 (2012) no. 3, pp. 681-701 | Zbl 1252.62080

[31] Roustant, Olivier; Ginsbourger, David; Deville, Yves Dicekriging, Diceoptim: Two R packages for the analysis of computer experiments by kriging-based metamodelling and optimization, Journal of Statistical Software, Volume 51 (2012) no. 1, 54 pages | Article

[32] Roache, Patrick J Verification of codes and calculations, AIAA journal, Volume 36 (1998) no. 5, pp. 696-702

[33] Robert, Christian Méthodes de Monte Carlo par chaînes de Markov, Economica, 1996 | Zbl 0917.60007

[34] Rocquigny, E de Quantifying uncertainty in an industrial approach: an emerging consensus in an old epistemological debate, SAPI EN. S. Surveys and Perspectives Integrating Environment and Society (2009) no. 2.1

[35] Sacks, Jerome; Welch, William J; Mitchell, Toby J; Wynn, Henry P Design and analysis of computer experiments, Statistical science (1989), pp. 409-423 | Zbl 0955.62619

[36] Santner, Thomas J; Williams, Brian J; Notz, William I The design and analysis of computer experiments, Springer Science & Business Media, 2013

[37] Tuo, Rui; Wu, CF Jeff Efficient calibration for imperfect computer models, The Annals of Statistics, Volume 43 (2015) no. 6, pp. 2331-2352 | Zbl 1326.62228

[38] Tuo, Rui; Wu, J. A theoretical framework for calibration in computer models: parametrization, estimation and convergence properties, SIAM/ASA Journal on Uncertainty Quantification, Volume 4 (2016) no. 1, pp. 767-795 | Zbl 1383.62354

[39] Wong, Raymond KW; Storlie, Curtis B; Lee, Thomas A frequentist approach to computer model calibration, Journal of the Royal Statistical Society: Series B (Statistical Methodology), Volume 79 (2017) no. 2, pp. 635-648 | Zbl 1414.62079