Multiclass ROC analysis

Matthew S. Wandishin, Steven J. Mullen

Research output: Contribution to journalArticlepeer-review

12 Scopus citations

Abstract

Receiver operating characteristic (ROC) curves have become a common analysis tool for evaluating forecast discrimination: the ability of a forecast system to distinguish between events and nonevents. As is implicit in that statement, application of the ROC curve is limited to forecasts involving only two possible outcomes, such as rain and no rain. However, many forecast scenarios exist for which there are multiple possible outcomes, such as rain, snow, and freezing rain. An extension of the ROC curve to multiclass forecast problems is explored. The full extension involves high-dimensional hypersurfaces that cannot be visualized and that present other problems. Therefore, several different approximations to the full extension are introduced using both artificial and actual forecast datasets. These approximations range from sets of simple two-class ROC curves to sets of three-dimensional ROC surfaces. No single approximation is superior for all forecast problems; thus, the specific aims in evaluating the forecast must be considered.

Original languageEnglish (US)
Pages (from-to)530-547
Number of pages18
JournalWeather and Forecasting
Volume24
Issue number2
DOIs
StatePublished - Apr 2009

ASJC Scopus subject areas

  • Atmospheric Science

Fingerprint

Dive into the research topics of 'Multiclass ROC analysis'. Together they form a unique fingerprint.

Cite this