On the optimal design of experiments for conceptual and predictive discrimination of hydrologic system models

C. P. Kikuchi, T. P.A. Ferré, J. A. Vrugt

Research output: Contribution to journalArticle

19 Scopus citations

Abstract

Experimental design and data collection constitute two main steps of the iterative research cycle (aka the scientific method). To help evaluate competing hypotheses, it is critical to ensure that the experimental design is appropriate and maximizes information retrieval from the system of interest. Scientific hypothesis testing is implemented by comparing plausible model structures (conceptual discrimination) and sets of predictions (predictive discrimination). This research presents a new Discrimination-Inference (DI) methodology to identify prospective data sets highly suitable for either conceptual or predictive discrimination. The DI methodology uses preposterior estimation techniques to evaluate the expected change in the conceptual or predictive probabilities, as measured by the Kullback-Leibler divergence. We present two case studies with increasing complexity to illustrate implementation of the DI for maximizing information withdrawal from a system of interest. The case studies show that highly informative data sets for conceptual discrimination are in general those for which between-model (conceptual) uncertainty is large relative to the within-model (parameter) uncertainty, and the redundancy between individual measurements in the set is minimized. The optimal data set differs if predictive, rather than conceptual, discrimination is the experimental design objective. Our results show that DI analyses highlight measurements that can be used to address critical uncertainties related to the prediction of interest. Finally, we find that the optimal data set for predictive discrimination is sensitive to the predictive grouping definition in ways that are not immediately apparent from inspection of the model structure and parameter values. Key Points: We present a metric to quantify conceptual and predictive discrimination The expected discrimination of prospective data points are calculated Optimal data points depend on between-model variance and information redundancy

Original languageEnglish (US)
Pages (from-to)4454-4481
Number of pages28
JournalWater Resources Research
Volume51
Issue number6
DOIs
StatePublished - Jun 1 2015

Keywords

  • experimental design
  • groundwater modeling
  • information theory
  • model discrimination

ASJC Scopus subject areas

  • Water Science and Technology

Fingerprint Dive into the research topics of 'On the optimal design of experiments for conceptual and predictive discrimination of hydrologic system models'. Together they form a unique fingerprint.

  • Cite this