Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling

Hoshin Vijai Gupta, Harald Kling, Koray K. Yilmaz, Guillermo F. Martinez

Research output: Contribution to journalArticle

958 Citations (Scopus)

Abstract

The mean squared error (MSE) and the related normalization, the Nash-Sutcliffe efficiency (NSE), are the two criteria most widely used for calibration and evaluation of hydrological models with observed data. Here, we present a diagnostically interesting decomposition of NSE (and hence MSE), which facilitates analysis of the relative importance of its different components in the context of hydrological modelling, and show how model calibration problems can arise due to interactions among these components. The analysis is illustrated by calibrating a simple conceptual precipitation-runoff model to daily data for a number of Austrian basins having a broad range of hydro-meteorological characteristics. Evaluation of the results clearly demonstrates the problems that can be associated with any calibration based on the NSE (or MSE) criterion. While we propose and test an alternative criterion that can help to reduce model calibration problems, the primary purpose of this study is not to present an improved measure of model performance. Instead, we seek to show that there are systematic problems inherent with any optimization based on formulations related to the MSE. The analysis and results have implications to the manner in which we calibrate and evaluate environmental models; we discuss these and suggest possible ways forward that may move us towards an improved and diagnostically meaningful approach to model performance evaluation and identification.

Original languageEnglish (US)
Pages (from-to)80-91
Number of pages12
JournalJournal of Hydrology
Volume377
Issue number1-2
DOIs
StatePublished - Oct 20 2009

Fingerprint

hydrological modeling
decomposition
calibration
runoff
basin
evaluation
analysis

Keywords

  • Calibration
  • Criteria decomposition
  • Mean squared error
  • Model performance evaluation
  • Multiple criteria
  • Nash-Sutcliffe efficiency

ASJC Scopus subject areas

  • Water Science and Technology

Cite this

Decomposition of the mean squared error and NSE performance criteria : Implications for improving hydrological modelling. / Gupta, Hoshin Vijai; Kling, Harald; Yilmaz, Koray K.; Martinez, Guillermo F.

In: Journal of Hydrology, Vol. 377, No. 1-2, 20.10.2009, p. 80-91.

Research output: Contribution to journalArticle

Gupta, Hoshin Vijai ; Kling, Harald ; Yilmaz, Koray K. ; Martinez, Guillermo F. / Decomposition of the mean squared error and NSE performance criteria : Implications for improving hydrological modelling. In: Journal of Hydrology. 2009 ; Vol. 377, No. 1-2. pp. 80-91.
@article{f5ec8471169f461288fbe55b0248a534,
title = "Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling",
abstract = "The mean squared error (MSE) and the related normalization, the Nash-Sutcliffe efficiency (NSE), are the two criteria most widely used for calibration and evaluation of hydrological models with observed data. Here, we present a diagnostically interesting decomposition of NSE (and hence MSE), which facilitates analysis of the relative importance of its different components in the context of hydrological modelling, and show how model calibration problems can arise due to interactions among these components. The analysis is illustrated by calibrating a simple conceptual precipitation-runoff model to daily data for a number of Austrian basins having a broad range of hydro-meteorological characteristics. Evaluation of the results clearly demonstrates the problems that can be associated with any calibration based on the NSE (or MSE) criterion. While we propose and test an alternative criterion that can help to reduce model calibration problems, the primary purpose of this study is not to present an improved measure of model performance. Instead, we seek to show that there are systematic problems inherent with any optimization based on formulations related to the MSE. The analysis and results have implications to the manner in which we calibrate and evaluate environmental models; we discuss these and suggest possible ways forward that may move us towards an improved and diagnostically meaningful approach to model performance evaluation and identification.",
keywords = "Calibration, Criteria decomposition, Mean squared error, Model performance evaluation, Multiple criteria, Nash-Sutcliffe efficiency",
author = "Gupta, {Hoshin Vijai} and Harald Kling and Yilmaz, {Koray K.} and Martinez, {Guillermo F.}",
year = "2009",
month = "10",
day = "20",
doi = "10.1016/j.jhydrol.2009.08.003",
language = "English (US)",
volume = "377",
pages = "80--91",
journal = "Journal of Hydrology",
issn = "0022-1694",
publisher = "Elsevier",
number = "1-2",

}

TY - JOUR

T1 - Decomposition of the mean squared error and NSE performance criteria

T2 - Implications for improving hydrological modelling

AU - Gupta, Hoshin Vijai

AU - Kling, Harald

AU - Yilmaz, Koray K.

AU - Martinez, Guillermo F.

PY - 2009/10/20

Y1 - 2009/10/20

N2 - The mean squared error (MSE) and the related normalization, the Nash-Sutcliffe efficiency (NSE), are the two criteria most widely used for calibration and evaluation of hydrological models with observed data. Here, we present a diagnostically interesting decomposition of NSE (and hence MSE), which facilitates analysis of the relative importance of its different components in the context of hydrological modelling, and show how model calibration problems can arise due to interactions among these components. The analysis is illustrated by calibrating a simple conceptual precipitation-runoff model to daily data for a number of Austrian basins having a broad range of hydro-meteorological characteristics. Evaluation of the results clearly demonstrates the problems that can be associated with any calibration based on the NSE (or MSE) criterion. While we propose and test an alternative criterion that can help to reduce model calibration problems, the primary purpose of this study is not to present an improved measure of model performance. Instead, we seek to show that there are systematic problems inherent with any optimization based on formulations related to the MSE. The analysis and results have implications to the manner in which we calibrate and evaluate environmental models; we discuss these and suggest possible ways forward that may move us towards an improved and diagnostically meaningful approach to model performance evaluation and identification.

AB - The mean squared error (MSE) and the related normalization, the Nash-Sutcliffe efficiency (NSE), are the two criteria most widely used for calibration and evaluation of hydrological models with observed data. Here, we present a diagnostically interesting decomposition of NSE (and hence MSE), which facilitates analysis of the relative importance of its different components in the context of hydrological modelling, and show how model calibration problems can arise due to interactions among these components. The analysis is illustrated by calibrating a simple conceptual precipitation-runoff model to daily data for a number of Austrian basins having a broad range of hydro-meteorological characteristics. Evaluation of the results clearly demonstrates the problems that can be associated with any calibration based on the NSE (or MSE) criterion. While we propose and test an alternative criterion that can help to reduce model calibration problems, the primary purpose of this study is not to present an improved measure of model performance. Instead, we seek to show that there are systematic problems inherent with any optimization based on formulations related to the MSE. The analysis and results have implications to the manner in which we calibrate and evaluate environmental models; we discuss these and suggest possible ways forward that may move us towards an improved and diagnostically meaningful approach to model performance evaluation and identification.

KW - Calibration

KW - Criteria decomposition

KW - Mean squared error

KW - Model performance evaluation

KW - Multiple criteria

KW - Nash-Sutcliffe efficiency

UR - http://www.scopus.com/inward/record.url?scp=70349272865&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70349272865&partnerID=8YFLogxK

U2 - 10.1016/j.jhydrol.2009.08.003

DO - 10.1016/j.jhydrol.2009.08.003

M3 - Article

AN - SCOPUS:70349272865

VL - 377

SP - 80

EP - 91

JO - Journal of Hydrology

JF - Journal of Hydrology

SN - 0022-1694

IS - 1-2

ER -