Hard or soft classification? large-margin unified machines

Yufeng Liu, Hao Helen Zhang, Yichao Wu

Research output: Contribution to journalArticle

50 Scopus citations

Abstract

Margin-based classifiers have been popular in both machine learning and statistics for classification problems. Among numerous classifiers, some are hard classifiers while some are soft ones. Soft classifiers explicitly estimate the class conditional probabilities and then perform classification based on estimated probabilities. In contrast, hard classifiers directly target the classification decision boundary without producing the probability estimation. These two types of classifiers are based on different philosophies and each has its own merits. In this article, we propose a novel family of large-margin classifiers, namely large-margin unified machines (LUMs), which covers a broad range of margin-based classifiers including both hard and soft ones. By offering a natural bridge from soft to hard classification, the LUM provides a unified algorithm to fit various classifiers and hence a convenient platform to compare hard and soft classification. Both theoretical consistency and numerical performance of LUMs are explored. Our numerical study sheds some light on the choice between hard and soft classifiers in various classification problems.

Original languageEnglish (US)
Pages (from-to)166-177
Number of pages12
JournalJournal of the American Statistical Association
Volume106
Issue number493
DOIs
StatePublished - Mar 1 2011
Externally publishedYes

Keywords

  • Class probability estimation
  • DWD
  • Fisher consistency
  • Regularization
  • SVM

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint Dive into the research topics of 'Hard or soft classification? large-margin unified machines'. Together they form a unique fingerprint.

  • Cite this