Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models

Hoshin Vijai Gupta, Saman Razavi

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

This paper investigates the problem of global sensitivity analysis (GSA) of Dynamical Earth System Models and proposes a basis for how such analyses should be performed. We argue that (a) performance metric-based approaches to parameter GSA are actually identifiability analyses, (b) the use of a performance metric to assess sensitivity unavoidably distorts the information provided by the model about relative parameter importance, and (c) it is a serious conceptual flaw to interpret the results of such an analysis as being consistent and accurate indications of the sensitivity of the model response to parameter perturbations. Further, because such approaches depend on availability of system state/output observational data, the analysis they provide is necessarily incomplete. Here we frame the GSA problem from first principles, using trajectories of the partial derivatives of model outputs with respect to controlling factors as the theoretical basis for sensitivity, and construct a global sensitivity matrix from which statistical indices of total period time-aggregate parameter importance, and time series of time-varying parameter importance, can be inferred. We demonstrate this framework using the HBV-SASK conceptual hydrologic model applied to the Oldman basin in Canada and show that it disagrees with performance metric-based methods regarding which parameters exert the strongest controls on model behavior. Further, it is highly efficient, requiring less than 1,000 base samples to obtain stable and robust parameter importance assessments for our 10-parameter example.

Original languageEnglish (US)
JournalWater Resources Research
DOIs
StateAccepted/In press - Jan 1 2018

Fingerprint

sensitivity analysis
parameter
trajectory
perturbation
time series
matrix
basin

Keywords

  • dynamical systems
  • efficiency and robustness
  • global sensitivity analysis
  • global sensitivity matrix
  • Parameter importance analysis
  • time-varying sensitivity

ASJC Scopus subject areas

  • Water Science and Technology

Cite this

Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models. / Gupta, Hoshin Vijai; Razavi, Saman.

In: Water Resources Research, 01.01.2018.

Research output: Contribution to journalArticle

@article{1df2364b0d4a4952ae786c7032520bd2,
title = "Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models",
abstract = "This paper investigates the problem of global sensitivity analysis (GSA) of Dynamical Earth System Models and proposes a basis for how such analyses should be performed. We argue that (a) performance metric-based approaches to parameter GSA are actually identifiability analyses, (b) the use of a performance metric to assess sensitivity unavoidably distorts the information provided by the model about relative parameter importance, and (c) it is a serious conceptual flaw to interpret the results of such an analysis as being consistent and accurate indications of the sensitivity of the model response to parameter perturbations. Further, because such approaches depend on availability of system state/output observational data, the analysis they provide is necessarily incomplete. Here we frame the GSA problem from first principles, using trajectories of the partial derivatives of model outputs with respect to controlling factors as the theoretical basis for sensitivity, and construct a global sensitivity matrix from which statistical indices of total period time-aggregate parameter importance, and time series of time-varying parameter importance, can be inferred. We demonstrate this framework using the HBV-SASK conceptual hydrologic model applied to the Oldman basin in Canada and show that it disagrees with performance metric-based methods regarding which parameters exert the strongest controls on model behavior. Further, it is highly efficient, requiring less than 1,000 base samples to obtain stable and robust parameter importance assessments for our 10-parameter example.",
keywords = "dynamical systems, efficiency and robustness, global sensitivity analysis, global sensitivity matrix, Parameter importance analysis, time-varying sensitivity",
author = "Gupta, {Hoshin Vijai} and Saman Razavi",
year = "2018",
month = "1",
day = "1",
doi = "10.1029/2018WR022668",
language = "English (US)",
journal = "Water Resources Research",
issn = "0043-1397",
publisher = "American Geophysical Union",

}

TY - JOUR

T1 - Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models

AU - Gupta, Hoshin Vijai

AU - Razavi, Saman

PY - 2018/1/1

Y1 - 2018/1/1

N2 - This paper investigates the problem of global sensitivity analysis (GSA) of Dynamical Earth System Models and proposes a basis for how such analyses should be performed. We argue that (a) performance metric-based approaches to parameter GSA are actually identifiability analyses, (b) the use of a performance metric to assess sensitivity unavoidably distorts the information provided by the model about relative parameter importance, and (c) it is a serious conceptual flaw to interpret the results of such an analysis as being consistent and accurate indications of the sensitivity of the model response to parameter perturbations. Further, because such approaches depend on availability of system state/output observational data, the analysis they provide is necessarily incomplete. Here we frame the GSA problem from first principles, using trajectories of the partial derivatives of model outputs with respect to controlling factors as the theoretical basis for sensitivity, and construct a global sensitivity matrix from which statistical indices of total period time-aggregate parameter importance, and time series of time-varying parameter importance, can be inferred. We demonstrate this framework using the HBV-SASK conceptual hydrologic model applied to the Oldman basin in Canada and show that it disagrees with performance metric-based methods regarding which parameters exert the strongest controls on model behavior. Further, it is highly efficient, requiring less than 1,000 base samples to obtain stable and robust parameter importance assessments for our 10-parameter example.

AB - This paper investigates the problem of global sensitivity analysis (GSA) of Dynamical Earth System Models and proposes a basis for how such analyses should be performed. We argue that (a) performance metric-based approaches to parameter GSA are actually identifiability analyses, (b) the use of a performance metric to assess sensitivity unavoidably distorts the information provided by the model about relative parameter importance, and (c) it is a serious conceptual flaw to interpret the results of such an analysis as being consistent and accurate indications of the sensitivity of the model response to parameter perturbations. Further, because such approaches depend on availability of system state/output observational data, the analysis they provide is necessarily incomplete. Here we frame the GSA problem from first principles, using trajectories of the partial derivatives of model outputs with respect to controlling factors as the theoretical basis for sensitivity, and construct a global sensitivity matrix from which statistical indices of total period time-aggregate parameter importance, and time series of time-varying parameter importance, can be inferred. We demonstrate this framework using the HBV-SASK conceptual hydrologic model applied to the Oldman basin in Canada and show that it disagrees with performance metric-based methods regarding which parameters exert the strongest controls on model behavior. Further, it is highly efficient, requiring less than 1,000 base samples to obtain stable and robust parameter importance assessments for our 10-parameter example.

KW - dynamical systems

KW - efficiency and robustness

KW - global sensitivity analysis

KW - global sensitivity matrix

KW - Parameter importance analysis

KW - time-varying sensitivity

UR - http://www.scopus.com/inward/record.url?scp=85056162545&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056162545&partnerID=8YFLogxK

U2 - 10.1029/2018WR022668

DO - 10.1029/2018WR022668

M3 - Article

AN - SCOPUS:85056162545

JO - Water Resources Research

JF - Water Resources Research

SN - 0043-1397

ER -