Multiclass proximal support vector machines

Yongqiang Tang, Hao Zhang

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

This article proposes the multiclass proximal support vector machine (MPSVM) classifier, which extends the binary PSVM to the multiclass case. Unlike the one-versus-rest approach that constructs the decision rule based on multiple binary classification tasks, the proposed method considers all classes simultaneously and has better theoretical properties and empirical performance. We formulate the MPSVM as a regularization problem in the reproducing kernel Hilbert space and show that it implements the Bayes rule for classification. In addition, the MPSVM can handle equal and unequal misclassification costs in a unified framework. We suggest an efficient algorithm to implement the MPSVM by solving a system of linear equations. This algorithm requires much less computational effort than solving the standard SVM, which often requires quadratic programming and can be slow for large problems. We also provide an alternative and more robust algorithm for ill-posed problems. The effectiveness of the MPSVM is demonstrated by both simulation studies and applications to cancer classifications using microarray data.

Original languageEnglish (US)
Pages (from-to)339-355
Number of pages17
JournalJournal of Computational and Graphical Statistics
Volume15
Issue number2
DOIs
StatePublished - Jun 2006
Externally publishedYes

Fingerprint

Multi-class
Support vector machines
Support Vector Machine
Cancer Classification
Quadratic programming
Hilbert spaces
Bayes Rule
Microarrays
Linear equations
Binary Classification
Reproducing Kernel Hilbert Space
Misclassification
Robust Algorithm
Ill-posed Problem
Decision Rules
System of Linear Equations
Microarray Data
Quadratic Programming
Unequal
Classifiers

Keywords

  • Bayes rule
  • Nonstandard classifications
  • Reproducing kernel Hilbert space

ASJC Scopus subject areas

  • Mathematics(all)
  • Statistics and Probability
  • Computational Mathematics

Cite this

Multiclass proximal support vector machines. / Tang, Yongqiang; Zhang, Hao.

In: Journal of Computational and Graphical Statistics, Vol. 15, No. 2, 06.2006, p. 339-355.

Research output: Contribution to journalArticle

@article{74bbf160efd341d48bb65ded52c568c3,
title = "Multiclass proximal support vector machines",
abstract = "This article proposes the multiclass proximal support vector machine (MPSVM) classifier, which extends the binary PSVM to the multiclass case. Unlike the one-versus-rest approach that constructs the decision rule based on multiple binary classification tasks, the proposed method considers all classes simultaneously and has better theoretical properties and empirical performance. We formulate the MPSVM as a regularization problem in the reproducing kernel Hilbert space and show that it implements the Bayes rule for classification. In addition, the MPSVM can handle equal and unequal misclassification costs in a unified framework. We suggest an efficient algorithm to implement the MPSVM by solving a system of linear equations. This algorithm requires much less computational effort than solving the standard SVM, which often requires quadratic programming and can be slow for large problems. We also provide an alternative and more robust algorithm for ill-posed problems. The effectiveness of the MPSVM is demonstrated by both simulation studies and applications to cancer classifications using microarray data.",
keywords = "Bayes rule, Nonstandard classifications, Reproducing kernel Hilbert space",
author = "Yongqiang Tang and Hao Zhang",
year = "2006",
month = "6",
doi = "10.1198/106186006X113647",
language = "English (US)",
volume = "15",
pages = "339--355",
journal = "Journal of Computational and Graphical Statistics",
issn = "1061-8600",
publisher = "American Statistical Association",
number = "2",

}

TY - JOUR

T1 - Multiclass proximal support vector machines

AU - Tang, Yongqiang

AU - Zhang, Hao

PY - 2006/6

Y1 - 2006/6

N2 - This article proposes the multiclass proximal support vector machine (MPSVM) classifier, which extends the binary PSVM to the multiclass case. Unlike the one-versus-rest approach that constructs the decision rule based on multiple binary classification tasks, the proposed method considers all classes simultaneously and has better theoretical properties and empirical performance. We formulate the MPSVM as a regularization problem in the reproducing kernel Hilbert space and show that it implements the Bayes rule for classification. In addition, the MPSVM can handle equal and unequal misclassification costs in a unified framework. We suggest an efficient algorithm to implement the MPSVM by solving a system of linear equations. This algorithm requires much less computational effort than solving the standard SVM, which often requires quadratic programming and can be slow for large problems. We also provide an alternative and more robust algorithm for ill-posed problems. The effectiveness of the MPSVM is demonstrated by both simulation studies and applications to cancer classifications using microarray data.

AB - This article proposes the multiclass proximal support vector machine (MPSVM) classifier, which extends the binary PSVM to the multiclass case. Unlike the one-versus-rest approach that constructs the decision rule based on multiple binary classification tasks, the proposed method considers all classes simultaneously and has better theoretical properties and empirical performance. We formulate the MPSVM as a regularization problem in the reproducing kernel Hilbert space and show that it implements the Bayes rule for classification. In addition, the MPSVM can handle equal and unequal misclassification costs in a unified framework. We suggest an efficient algorithm to implement the MPSVM by solving a system of linear equations. This algorithm requires much less computational effort than solving the standard SVM, which often requires quadratic programming and can be slow for large problems. We also provide an alternative and more robust algorithm for ill-posed problems. The effectiveness of the MPSVM is demonstrated by both simulation studies and applications to cancer classifications using microarray data.

KW - Bayes rule

KW - Nonstandard classifications

KW - Reproducing kernel Hilbert space

UR - http://www.scopus.com/inward/record.url?scp=33745305029&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33745305029&partnerID=8YFLogxK

U2 - 10.1198/106186006X113647

DO - 10.1198/106186006X113647

M3 - Article

AN - SCOPUS:33745305029

VL - 15

SP - 339

EP - 355

JO - Journal of Computational and Graphical Statistics

JF - Journal of Computational and Graphical Statistics

SN - 1061-8600

IS - 2

ER -