How certain are our uncertainty bounds? Accounting for sample variability in Monte Carlo-based uncertainty estimates

Tirthankar Roy, Hoshin Gupta

Research output: Contribution to journalArticlepeer-review

Abstract

It is common for model-based simulations to be reported using prediction interval estimates that characterize the lack of precision associated with the simulated values. When based on Monte-Carlo sampling to approximate the relevant probability density function(s), such estimates can significantly underestimate the width of the prediction intervals, unless the sample size is sufficiently large. Using theoretical arguments supported by numerical experiments, we discuss the nature and severity of this problem, and demonstrate how better estimates of prediction intervals can be achieved by adjusting the interval width to account for the size of the sample used in its construction. Our method is generally applicable regardless of the form of the underlying probability density function, and can be particularly useful when the model is expensive to run and large samples are not available. We illustrate its use via a simple example involving conceptual modeling of the rainfall-runoff response of a catchment.

Original languageEnglish (US)
Article number104931
JournalEnvironmental Modelling and Software
Volume136
DOIs
StatePublished - Feb 2021

Keywords

  • Estimation
  • Monte Carlo simulation
  • Precision
  • Prediction intervals
  • Sampling variability
  • Uncertainty

ASJC Scopus subject areas

  • Software
  • Environmental Engineering
  • Ecological Modeling

Fingerprint Dive into the research topics of 'How certain are our uncertainty bounds? Accounting for sample variability in Monte Carlo-based uncertainty estimates'. Together they form a unique fingerprint.

Cite this