Abstract
Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 424-436 |
| Number of pages | 13 |
| Journal | Journal. American Statistical Association |
| Volume | 105 |
| Issue number | 489 |
| DOIs | |
| State | Published - Mar 2010 |
| Externally published | Yes |
Fingerprint
Keywords
- Fisher consistency
- Hard classification
- Multicategory classification
- Probability estimation
- Soft classification
- SVM
ASJC Scopus subject areas
- Statistics and Probability
- Statistics, Probability and Uncertainty
Cite this
Robust model-free multiclass probability estimation. / Wu, Yichao; Zhang, Hao; Liu, Yufeng.
In: Journal. American Statistical Association, Vol. 105, No. 489, 03.2010, p. 424-436.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Robust model-free multiclass probability estimation
AU - Wu, Yichao
AU - Zhang, Hao
AU - Liu, Yufeng
PY - 2010/3
Y1 - 2010/3
N2 - Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.
AB - Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.
KW - Fisher consistency
KW - Hard classification
KW - Multicategory classification
KW - Probability estimation
KW - Soft classification
KW - SVM
UR - http://www.scopus.com/inward/record.url?scp=77952567231&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=77952567231&partnerID=8YFLogxK
U2 - 10.1198/jasa.2010.tm09107
DO - 10.1198/jasa.2010.tm09107
M3 - Article
AN - SCOPUS:77952567231
VL - 105
SP - 424
EP - 436
JO - Journal of the American Statistical Association
JF - Journal of the American Statistical Association
SN - 0162-1459
IS - 489
ER -