Ensembles vs. information theory: supporting science under uncertainty

Grey S. Nearing, Hoshin Vijai Gupta

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Multi-model ensembles are one of the most common ways to deal with epistemic uncertainty in hydrology. This is a problem because there is no known way to sample models such that the resulting ensemble admits a measure that has any systematic (i.e., asymptotic, bounded, or consistent) relationship with uncertainty. Multi-model ensembles are effectively sensitivity analyses and cannot – even partially – quantify uncertainty. One consequence of this is that multi-model approaches cannot support a consistent scientific method – in particular, multi-model approaches yield unbounded errors in inference. In contrast, information theory supports a coherent hypothesis test that is robust to (i.e., bounded under) arbitrary epistemic uncertainty. This paper may be understood as advocating a procedure for hypothesis testing that does not require quantifying uncertainty, but is coherent and reliable (i.e., bounded) in the presence of arbitrary (unknown and unknowable) uncertainty. We conclude by offering some suggestions about how this proposed philosophy of science suggests new ways to conceptualize and construct simulation models of complex, dynamical systems.

Original languageEnglish (US)
Pages (from-to)1-8
Number of pages8
JournalFrontiers of Earth Science
DOIs
StateAccepted/In press - May 9 2018

Fingerprint

hypothesis testing
science
hydrology
simulation
method
test

Keywords

  • Bayesian methods
  • hypothesis testing
  • information theory
  • multi-model ensembles
  • uncertainty quantification

ASJC Scopus subject areas

  • Earth and Planetary Sciences(all)

Cite this

Ensembles vs. information theory : supporting science under uncertainty. / Nearing, Grey S.; Gupta, Hoshin Vijai.

In: Frontiers of Earth Science, 09.05.2018, p. 1-8.

Research output: Contribution to journalArticle

@article{602733951ea84ecd80ff420c3ffbf7bb,
title = "Ensembles vs. information theory: supporting science under uncertainty",
abstract = "Multi-model ensembles are one of the most common ways to deal with epistemic uncertainty in hydrology. This is a problem because there is no known way to sample models such that the resulting ensemble admits a measure that has any systematic (i.e., asymptotic, bounded, or consistent) relationship with uncertainty. Multi-model ensembles are effectively sensitivity analyses and cannot – even partially – quantify uncertainty. One consequence of this is that multi-model approaches cannot support a consistent scientific method – in particular, multi-model approaches yield unbounded errors in inference. In contrast, information theory supports a coherent hypothesis test that is robust to (i.e., bounded under) arbitrary epistemic uncertainty. This paper may be understood as advocating a procedure for hypothesis testing that does not require quantifying uncertainty, but is coherent and reliable (i.e., bounded) in the presence of arbitrary (unknown and unknowable) uncertainty. We conclude by offering some suggestions about how this proposed philosophy of science suggests new ways to conceptualize and construct simulation models of complex, dynamical systems.",
keywords = "Bayesian methods, hypothesis testing, information theory, multi-model ensembles, uncertainty quantification",
author = "Nearing, {Grey S.} and Gupta, {Hoshin Vijai}",
year = "2018",
month = "5",
day = "9",
doi = "10.1007/s11707-018-0709-9",
language = "English (US)",
pages = "1--8",
journal = "Frontiers of Earth Science",
issn = "1673-7385",
publisher = "Higher Education Press",

}

TY - JOUR

T1 - Ensembles vs. information theory

T2 - supporting science under uncertainty

AU - Nearing, Grey S.

AU - Gupta, Hoshin Vijai

PY - 2018/5/9

Y1 - 2018/5/9

N2 - Multi-model ensembles are one of the most common ways to deal with epistemic uncertainty in hydrology. This is a problem because there is no known way to sample models such that the resulting ensemble admits a measure that has any systematic (i.e., asymptotic, bounded, or consistent) relationship with uncertainty. Multi-model ensembles are effectively sensitivity analyses and cannot – even partially – quantify uncertainty. One consequence of this is that multi-model approaches cannot support a consistent scientific method – in particular, multi-model approaches yield unbounded errors in inference. In contrast, information theory supports a coherent hypothesis test that is robust to (i.e., bounded under) arbitrary epistemic uncertainty. This paper may be understood as advocating a procedure for hypothesis testing that does not require quantifying uncertainty, but is coherent and reliable (i.e., bounded) in the presence of arbitrary (unknown and unknowable) uncertainty. We conclude by offering some suggestions about how this proposed philosophy of science suggests new ways to conceptualize and construct simulation models of complex, dynamical systems.

AB - Multi-model ensembles are one of the most common ways to deal with epistemic uncertainty in hydrology. This is a problem because there is no known way to sample models such that the resulting ensemble admits a measure that has any systematic (i.e., asymptotic, bounded, or consistent) relationship with uncertainty. Multi-model ensembles are effectively sensitivity analyses and cannot – even partially – quantify uncertainty. One consequence of this is that multi-model approaches cannot support a consistent scientific method – in particular, multi-model approaches yield unbounded errors in inference. In contrast, information theory supports a coherent hypothesis test that is robust to (i.e., bounded under) arbitrary epistemic uncertainty. This paper may be understood as advocating a procedure for hypothesis testing that does not require quantifying uncertainty, but is coherent and reliable (i.e., bounded) in the presence of arbitrary (unknown and unknowable) uncertainty. We conclude by offering some suggestions about how this proposed philosophy of science suggests new ways to conceptualize and construct simulation models of complex, dynamical systems.

KW - Bayesian methods

KW - hypothesis testing

KW - information theory

KW - multi-model ensembles

KW - uncertainty quantification

UR - http://www.scopus.com/inward/record.url?scp=85046619521&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046619521&partnerID=8YFLogxK

U2 - 10.1007/s11707-018-0709-9

DO - 10.1007/s11707-018-0709-9

M3 - Article

AN - SCOPUS:85046619521

SP - 1

EP - 8

JO - Frontiers of Earth Science

JF - Frontiers of Earth Science

SN - 1673-7385

ER -