RobustCalibration: Robust Calibration of Computer Models in R

University of California, Santa Barbara

mengyang@pstat.ucsb.edu

Abstract

Two fundamental research tasks in science and engineering are forward predictions and data inversion. This article introduces a new R package RobustCalibration for Bayesian data inversion and model calibration using experiments and field observations. Mathematical models for forward predictions are often written in computer code, and they can be computationally expensive to run. To overcome the computational bottleneck from the simulator, we implemented a statistical emulator from the RobustGaSP package for emulating both scalar-valued or vector-valued computer model outputs. Both posterior sampling and maximum likelihood approach are implemented in the RobustCalibration package for parameter estimation. For imperfect computer models, we implement the Gaussian stochastic process and scaled Gaussian stochastic process for modeling the discrepancy function between the reality and mathematical model. This package is applicable to various other types of field observations and models, such as repeated experiments, multiple sources of measurements and correlated measurement bias. We discuss numerical examples of calibrating mathematical models that have closed-form expressions, and differential equations solved by numerical methods.

Agram, PS, and M Simons. 2015. “A Noise Model for InSAR Time Series.” Journal of Geophysical Research: Solid Earth 120 (4): 2752–71.

Anderson, Kyle R, Ingrid A Johanson, Matthew R Patrick, Mengyang Gu, Paul Segall, Michael P Poland, Emily K Montgomery-Brown, and Asta Miklius. 2019. “Magma Reservoir Failure and the Onset of Caldera Collapse at Kı̄lauea Volcano in 2018.” Science 366 (6470).

Arendt, Paul D, Daniel W Apley, and Wei Chen. 2012. “Quantification of Model Uncertainty: Calibration, Model Discrepancy, and Identifiability.” Journal of Mechanical Design 134 (10): 100908.

Bayarri, Maria J, James O Berger, Rui Paulo, Jerry Sacks, John A Cafeo, James Cavendish, Chin-Hsu Lin, and Jian Tu. 2007. “A Framework for Validation of Computer Models.” Technometrics 49 (2): 138–54.

Bayarri, MJ, JO Berger, J Cafeo, G Garcia-Donato, F Liu, J Palomo, RJ Parthasarathy, R Paulo, J Sacks, and D Walsh. 2007. “Computer Model Validation with Functional Output.” The Annals of Statistics 35 (5): 1874–1906.

Box, GEP, and GA Coutie. 1956. “Application of Digital Computers in the Exploration of Functional Relationships.” Proceedings of the IEE-Part B: Radio and Electronic Engineering 103 (1S): 100–107.

Brajard, Julien, Alberto Carrassi, Marc Bocquet, and Laurent Bertino. 2020. “Combining Data Assimilation and Machine Learning to Emulate a Dynamical Model from Sparse and Noisy Observations: A Case Study with the Lorenz 96 Model.” Journal of Computational Science 44: 101171.

Carmassi, Mathieu, Pierre Barbillon, Matthieu Chiodetti, Merlin Keller, and Eric Parent. 2018. “CaliCo: A R Package for Bayesian Calibration.” arXiv Preprint arXiv:1808.01932.

Chang, Won, Murali Haran, Patrick Applegate, and David Pollard. 2016. “Calibrating an Ice Sheet Model Using High-Dimensional Binary Spatial Data.” Journal of the American Statistical Association 111 (513): 57–72.

Chang, Won, Bledar A Konomi, Georgios Karagiannis, Yawen Guan, and Murali Haran. 2022. “Ice Model Calibration Using Semicontinuous Spatial Data.” The Annals of Applied Statistics 16 (3): 1937–61.

Gu, Mengyang. 2019. “Jointly Robust Prior for Gaussian Stochastic Process in Emulation, Calibration and Variable Selection.” Bayesian Analysis 14 (1).

Gu, Mengyang, Kyle Anderson, and Erika McPhillips. 2023. “Calibration of Imperfect Geophysical Models by Multiple Satellite Interferograms with Measurement Bias.” Technometrics, In Press. https://doi.org/10.1080/00401706.2023.2182365.

Gu, Mengyang, Jesus Palomo, and James O. Berger. 2019. “RobustGaSP: Robust Gaussian Stochastic Process Emulation in R.” The R Journal 11 (1): 112–36. https://doi.org/10.32614/RJ-2019-011.

Gu, Mengyang, and Long Wang. 2018. “Scaled Gaussian Stochastic Process for Computer Model Calibration and Prediction.” SIAM/ASA Journal on Uncertainty Quantification 6 (4): 1555–83.

Gu, Mengyang, Fangzheng Xie, and Long Wang. 2022. “A Theoretical Framework of the Scaled Gaussian Stochastic Process in Prediction and Calibration.” SIAM/ASA Journal on Uncertainty Quantification 10 (4): 1435–60.

Hankin, Robin KS. 2005. “Introducing BACCO, an R Bundle for Bayesian Analysis of Computer Code Output.” Journal of Statistical Software 14: 1–21.

Higdon, Dave, James Gattiker, Brian Williams, and Maria Rightley. 2008. “Computer Model Calibration Using High-Dimensional Output.” Journal of the American Statistical Association 103 (482): 570–83.

Kennedy, Marc C, and Anthony O’Hagan. 2001. “Bayesian Calibration of Computer Models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 63 (3): 425–64.

Liu, Dong C, and Jorge Nocedal. 1989. “On the Limited Memory BFGS Method for Large Scale Optimization.” Mathematical Programming 45 (1-3): 503–28.

Liu, Fei, MJ Bayarri, and JO Berger. 2009. “Modularization in Bayesian Analysis, with Emphasis on Analysis of Computer Models.” Bayesian Analysis 4 (1): 119–50.

Lorenz, Edward N. 1996. “Predictability: A Problem Partly Solved.” In Proc. Seminar on Predictability. Vol. 1. 1.

MacDonald, Blake, Pritam Ranjan, Hugh Chipman, et al. 2015. “GPfit: An R Package for Fitting a Gaussian Process Model to Deterministic Simulator Outputs.” Journal of Statistical Software 64 (i12).

Maclean, John, and Elaine T Spiller. 2020. “A Surrogate-Based Approach to Nonlinear, Non-Gaussian Joint State-Parameter Data Assimilation.” arXiv Preprint arXiv:2012.04793.

Palomo, Jesús, Rui Paulo, Gonzalo Garcı́a-Donato, et al. 2015. “SAVE: An R Package for the Statistical Analysis of Computer Models.” Journal of Statistical Software 64 (13): 1–23.

Paulo, Rui, Gonzalo Garcı́a-Donato, and Jesús Palomo. 2012. “Calibration of Computer Models with Multivariate Output.” Computational Statistics and Data Analysis 56 (12): 3959–74.

Plumlee, Matthew. 2017. “Bayesian Calibration of Inexact Computer Models.” Journal of the American Statistical Association 112 (519): 1274–85.

Roustant, Olivier, David Ginsbourger, and Yves Deville. 2012. “DiceKriging, DiceOptim: Two R Packages for the Analysis of Computer Experiments by Kriging-Based Metamodeling and Optimization.” Journal of Statistical Software 51: 1–55.

Sacks, Jerome, William J Welch, Toby J Mitchell, Henry P Wynn, et al. 1989. “Design and Analysis of Computer Experiments.” Statistical Science 4 (4): 409–23.

Santner, Thomas J, Brian J Williams, and William I Notz. 2003. The Design and Analysis of Computer Experiments. Springer Science & Business Media.

Simakov, Nikolay A, Renette L Jones-Ivey, Ali Akhavan-Safaei, Hossein Aghakhani, Matthew D Jones, and Abani K Patra. 2019. “Modernizing Titan2D, a Parallel AMR Geophysical Flow Code to Support Multiple Rheologies and Extendability.” In International Conference on High Performance Computing, 101–12. Springer.

Soetaert, Karline, Thomas Petzoldt, and R Woodrow Setzer. 2010. “Solving Differential Equations in R: Package deSolve.” Journal of Statistical Software 33: 1–25.

Tuo, Rui, and CF Jeff Wu. 2015. “Efficient Calibration for Imperfect Computer Models.” The Annals of Statistics 43 (6): 2331–52.

Wickham, Hadley. 2011. “Ggplot2.” Wiley Interdisciplinary Reviews: Computational Statistics 3 (2): 180–85.

Williamson, Daniel, Michael Goldstein, Lesley Allison, Adam Blaker, Peter Challenor, Laura Jackson, and Kuniko Yamazaki. 2013. “History Matching for Exploring and Reducing Climate Model Parameter Space Using Observations and a Large Perturbed Physics Ensemble.” Climate Dynamics 41 (7-8): 1703–29.

Wong, Raymond KW, Curtis B Storlie, and Thomas Lee. 2017. “A Frequentist Approach to Computer Model Calibration.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 79: 635–48.

Ypma, Jelmer. 2014. Nloptr: R Interface to NLopt. https://CRAN.R-project.org/package=nloptr.

Zebker, Howard A., Paul A. Rosen, and Scott Hensley. 1997. “Atmospheric effects in interferometric synthetic aperture radar surface deformation and topographic maps.” Journal of Geophysical Research: Solid Earth 102 (B4): 7547–63. https://doi.org/10.1029/96JB03804.