Robust model-free multiclass probability estimation

Yichao Wu, Hao Zhang, Yufeng Liu

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.

Original languageEnglish (US)
Pages (from-to)424-436
Number of pages13
JournalJournal. American Statistical Association
Volume105
Issue number489
DOIs
StatePublished - Mar 2010
Externally publishedYes

Fingerprint

Multi-class
Margin
Discriminant Analysis
Classifier
Model
Regression Estimation
Probability function
Classification Rules
Computational Algorithm
Multiple Regression
Density Estimation
Logistic Regression
Estimate
Regression
Series
Range of data

Keywords

  • Fisher consistency
  • Hard classification
  • Multicategory classification
  • Probability estimation
  • Soft classification
  • SVM

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Robust model-free multiclass probability estimation. / Wu, Yichao; Zhang, Hao; Liu, Yufeng.

In: Journal. American Statistical Association, Vol. 105, No. 489, 03.2010, p. 424-436.

Research output: Contribution to journalArticle

Wu, Yichao ; Zhang, Hao ; Liu, Yufeng. / Robust model-free multiclass probability estimation. In: Journal. American Statistical Association. 2010 ; Vol. 105, No. 489. pp. 424-436.
@article{713cb032284a4783bbd46e593b6acdc1,
title = "Robust model-free multiclass probability estimation",
abstract = "Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.",
keywords = "Fisher consistency, Hard classification, Multicategory classification, Probability estimation, Soft classification, SVM",
author = "Yichao Wu and Hao Zhang and Yufeng Liu",
year = "2010",
month = "3",
doi = "10.1198/jasa.2010.tm09107",
language = "English (US)",
volume = "105",
pages = "424--436",
journal = "Journal of the American Statistical Association",
issn = "0162-1459",
publisher = "Taylor and Francis Ltd.",
number = "489",

}

TY - JOUR

T1 - Robust model-free multiclass probability estimation

AU - Wu, Yichao

AU - Zhang, Hao

AU - Liu, Yufeng

PY - 2010/3

Y1 - 2010/3

N2 - Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.

AB - Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.

KW - Fisher consistency

KW - Hard classification

KW - Multicategory classification

KW - Probability estimation

KW - Soft classification

KW - SVM

UR - http://www.scopus.com/inward/record.url?scp=77952567231&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77952567231&partnerID=8YFLogxK

U2 - 10.1198/jasa.2010.tm09107

DO - 10.1198/jasa.2010.tm09107

M3 - Article

AN - SCOPUS:77952567231

VL - 105

SP - 424

EP - 436

JO - Journal of the American Statistical Association

JF - Journal of the American Statistical Association

SN - 0162-1459

IS - 489

ER -