### Abstract

Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.

Original language | English (US) |
---|---|

Pages (from-to) | 424-436 |

Number of pages | 13 |

Journal | Journal. American Statistical Association |

Volume | 105 |

Issue number | 489 |

DOIs | |

State | Published - Mar 2010 |

Externally published | Yes |

### Fingerprint

### Keywords

- Fisher consistency
- Hard classification
- Multicategory classification
- Probability estimation
- Soft classification
- SVM

### ASJC Scopus subject areas

- Statistics and Probability
- Statistics, Probability and Uncertainty

### Cite this

*Journal. American Statistical Association*,

*105*(489), 424-436. https://doi.org/10.1198/jasa.2010.tm09107

**Robust model-free multiclass probability estimation.** / Wu, Yichao; Zhang, Hao; Liu, Yufeng.

Research output: Contribution to journal › Article

*Journal. American Statistical Association*, vol. 105, no. 489, pp. 424-436. https://doi.org/10.1198/jasa.2010.tm09107

}

TY - JOUR

T1 - Robust model-free multiclass probability estimation

AU - Wu, Yichao

AU - Zhang, Hao

AU - Liu, Yufeng

PY - 2010/3

Y1 - 2010/3

N2 - Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.

AB - Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.

KW - Fisher consistency

KW - Hard classification

KW - Multicategory classification

KW - Probability estimation

KW - Soft classification

KW - SVM

UR - http://www.scopus.com/inward/record.url?scp=77952567231&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77952567231&partnerID=8YFLogxK

U2 - 10.1198/jasa.2010.tm09107

DO - 10.1198/jasa.2010.tm09107

M3 - Article

AN - SCOPUS:77952567231

VL - 105

SP - 424

EP - 436

JO - Journal of the American Statistical Association

JF - Journal of the American Statistical Association

SN - 0162-1459

IS - 489

ER -