TY - JOUR

T1 - Ideal observers and optimal ROC hypersurfaces in N-class classification

AU - Edwards, Darrin C.

AU - Metz, Charles E.

AU - Kupinski, Matthew A.

N1 - Funding Information:
Manuscript received August 5, 2003; revised March 22, 2004. This work was supported in parts by the National Cancer Institute under Grant R01-CA60187 (R.M. Nishikawa, principal investigator) and in part by the National Institutes of Health under Grant R01-GM57622 (C. E. Metz, principal investigator). C. E. Metz is a shareholder in R2 Technology, Inc. (Sunnyvale, CA). The Associate Editor responsible for coordinating the review of this paper and recommending its publication was H.-P. Chan. Asterisk indicates corresponding author. *D. C. Edwards is with the Department of Radiology, the University of Chicago, Chicago, IL 60637 USA (e-mail: d-edwards@uchicago.edu).

PY - 2004/7

Y1 - 2004/7

N2 - The likelihood ratio, or ideal observer, decision rule is known to be optimal for two-class classification tasks in the sense that it maximizes expected utility (or, equivalently, minimizes the Bayes risk). Furthermore, using this decision rule yields a receiver operating characteristic (ROC) curve which is never above the ROC curve produced using any other decision rule, provided the observer's misclassification rate with respect to one of the two classes is chosen as the dependent variable for the curve (i.e., an "inversion" of the more common formulation in which the observer's true-positive fraction is plotted against its false-positive fraction). It is also known that for a decision task requiring classification of observations into N classes, optimal performance in the expected utility sense is obtained using a set of N - 1 likelihood ratios as decision variables. In the N-class extension of ROC analysis, the ideal observer performance is describable in terms of an (N2 - N - 1)-parameter hypersurface in an (N2 - N)-dimensional probability space. We show that the result for two classes holds in this case as well, namely that the ROC hypersurface obtained using the ideal observer decision rule is never above the ROC hypersurface obtained using any other decision rule (where in our formulation performance is given exclusively with respect to between-class error rates rather than within-class sensitivities).

AB - The likelihood ratio, or ideal observer, decision rule is known to be optimal for two-class classification tasks in the sense that it maximizes expected utility (or, equivalently, minimizes the Bayes risk). Furthermore, using this decision rule yields a receiver operating characteristic (ROC) curve which is never above the ROC curve produced using any other decision rule, provided the observer's misclassification rate with respect to one of the two classes is chosen as the dependent variable for the curve (i.e., an "inversion" of the more common formulation in which the observer's true-positive fraction is plotted against its false-positive fraction). It is also known that for a decision task requiring classification of observations into N classes, optimal performance in the expected utility sense is obtained using a set of N - 1 likelihood ratios as decision variables. In the N-class extension of ROC analysis, the ideal observer performance is describable in terms of an (N2 - N - 1)-parameter hypersurface in an (N2 - N)-dimensional probability space. We show that the result for two classes holds in this case as well, namely that the ROC hypersurface obtained using the ideal observer decision rule is never above the ROC hypersurface obtained using any other decision rule (where in our formulation performance is given exclusively with respect to between-class error rates rather than within-class sensitivities).

KW - Ideal observer

KW - N-class classification

KW - ROC analysis

UR - http://www.scopus.com/inward/record.url?scp=3142777803&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=3142777803&partnerID=8YFLogxK

U2 - 10.1109/TMI.2004.828358

DO - 10.1109/TMI.2004.828358

M3 - Article

C2 - 15250641

AN - SCOPUS:3142777803

VL - 23

SP - 891

EP - 895

JO - IEEE Transactions on Medical Imaging

JF - IEEE Transactions on Medical Imaging

SN - 0278-0062

IS - 7

ER -