Model performance, model robustness, and model fitness scores: A new method for identifying good land-surface models

Lindsey E. Gulden, Enrique Rosero, Zong Liang Yang, Thorsten Wagener, Guo-Yue Niu

Research output: Contribution to journalArticle

22 Citations (Scopus)

Abstract

We introduce three metrics for rigorous evaluation of land-surface models (LSMs). This framework explicitly acknowledges perennial sources of uncertainty in LSM output. The model performance score (ζ) quantifies the likelihood that a representative model ensemble will bracket most observations and be highly skilled with low spread. The robustness score (ρ) quantifies the sensitivity of performance to parameter and/or data error. The fitness score (φ) combines performance and robustness, ranking models' suitability for broad application. We demonstrate the use of the metrics by comparing three versions of the Noah LSM. Using time-varying ζ for hypothesis testing and model development, we show that representing short-term phenological change improves Noah's simulation of surface energy partitioning and subsurface water dynamics at a semi-humid site. The least complex version of Noah is most fit for broad application. The framework and metrics presented here can significantly improve the confidence that can be placed in LSM predictions.

Original languageEnglish (US)
Article numberL11404
JournalGeophysical Research Letters
Volume35
Issue number11
DOIs
StatePublished - Jun 16 2008
Externally publishedYes

Fingerprint

fitness
land surface
hypothesis testing
surface energy
method
ranking
brackets
partitioning
confidence
prediction
evaluation
output
sensitivity
simulation
predictions

ASJC Scopus subject areas

  • Earth and Planetary Sciences (miscellaneous)

Cite this

Model performance, model robustness, and model fitness scores : A new method for identifying good land-surface models. / Gulden, Lindsey E.; Rosero, Enrique; Yang, Zong Liang; Wagener, Thorsten; Niu, Guo-Yue.

In: Geophysical Research Letters, Vol. 35, No. 11, L11404, 16.06.2008.

Research output: Contribution to journalArticle

Gulden, Lindsey E. ; Rosero, Enrique ; Yang, Zong Liang ; Wagener, Thorsten ; Niu, Guo-Yue. / Model performance, model robustness, and model fitness scores : A new method for identifying good land-surface models. In: Geophysical Research Letters. 2008 ; Vol. 35, No. 11.
@article{a37c05a499aa41c69fd9de86a22a8299,
title = "Model performance, model robustness, and model fitness scores: A new method for identifying good land-surface models",
abstract = "We introduce three metrics for rigorous evaluation of land-surface models (LSMs). This framework explicitly acknowledges perennial sources of uncertainty in LSM output. The model performance score (ζ) quantifies the likelihood that a representative model ensemble will bracket most observations and be highly skilled with low spread. The robustness score (ρ) quantifies the sensitivity of performance to parameter and/or data error. The fitness score (φ) combines performance and robustness, ranking models' suitability for broad application. We demonstrate the use of the metrics by comparing three versions of the Noah LSM. Using time-varying ζ for hypothesis testing and model development, we show that representing short-term phenological change improves Noah's simulation of surface energy partitioning and subsurface water dynamics at a semi-humid site. The least complex version of Noah is most fit for broad application. The framework and metrics presented here can significantly improve the confidence that can be placed in LSM predictions.",
author = "Gulden, {Lindsey E.} and Enrique Rosero and Yang, {Zong Liang} and Thorsten Wagener and Guo-Yue Niu",
year = "2008",
month = "6",
day = "16",
doi = "10.1029/2008GL033721",
language = "English (US)",
volume = "35",
journal = "Geophysical Research Letters",
issn = "0094-8276",
publisher = "American Geophysical Union",
number = "11",

}

TY - JOUR

T1 - Model performance, model robustness, and model fitness scores

T2 - A new method for identifying good land-surface models

AU - Gulden, Lindsey E.

AU - Rosero, Enrique

AU - Yang, Zong Liang

AU - Wagener, Thorsten

AU - Niu, Guo-Yue

PY - 2008/6/16

Y1 - 2008/6/16

N2 - We introduce three metrics for rigorous evaluation of land-surface models (LSMs). This framework explicitly acknowledges perennial sources of uncertainty in LSM output. The model performance score (ζ) quantifies the likelihood that a representative model ensemble will bracket most observations and be highly skilled with low spread. The robustness score (ρ) quantifies the sensitivity of performance to parameter and/or data error. The fitness score (φ) combines performance and robustness, ranking models' suitability for broad application. We demonstrate the use of the metrics by comparing three versions of the Noah LSM. Using time-varying ζ for hypothesis testing and model development, we show that representing short-term phenological change improves Noah's simulation of surface energy partitioning and subsurface water dynamics at a semi-humid site. The least complex version of Noah is most fit for broad application. The framework and metrics presented here can significantly improve the confidence that can be placed in LSM predictions.

AB - We introduce three metrics for rigorous evaluation of land-surface models (LSMs). This framework explicitly acknowledges perennial sources of uncertainty in LSM output. The model performance score (ζ) quantifies the likelihood that a representative model ensemble will bracket most observations and be highly skilled with low spread. The robustness score (ρ) quantifies the sensitivity of performance to parameter and/or data error. The fitness score (φ) combines performance and robustness, ranking models' suitability for broad application. We demonstrate the use of the metrics by comparing three versions of the Noah LSM. Using time-varying ζ for hypothesis testing and model development, we show that representing short-term phenological change improves Noah's simulation of surface energy partitioning and subsurface water dynamics at a semi-humid site. The least complex version of Noah is most fit for broad application. The framework and metrics presented here can significantly improve the confidence that can be placed in LSM predictions.

UR - http://www.scopus.com/inward/record.url?scp=49849091344&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=49849091344&partnerID=8YFLogxK

U2 - 10.1029/2008GL033721

DO - 10.1029/2008GL033721

M3 - Article

AN - SCOPUS:49849091344

VL - 35

JO - Geophysical Research Letters

JF - Geophysical Research Letters

SN - 0094-8276

IS - 11

M1 - L11404

ER -