Variable selection for support vector machines via smoothing spline anova

Research output: Contribution to journalArticle

28 Scopus citations

Abstract

It is well-known that the support vector machine paradigm is equivalent to solving a regularization problem in a reproducing kernel Hubert space. The squared norm penalty in the standard support vector machine controls the smoothness of the classification function. We propose, under the framework of smoothing spline ANOVA models, a new type of regularization to conduct simultaneous classification and variable selection in the SVM. The penalty functional used is the sum of functional component norms, which automatically applies soft-thresholding operations to functional components, hence yields sparse solutions. We suggest an efficient algorithm to solve the proposed optimization problem by iteratively solving quadratic and linear programming problems. Numerical studies, on both simulated data and real datasets, show that the modified support vector machine gives very competitive performances compared to other popular classification algorithms, in terms of both classification accuracy and variable selection.

Original languageEnglish (US)
Pages (from-to)659-674
Number of pages16
JournalStatistica Sinica
Volume16
Issue number2
StatePublished - Apr 1 2006
Externally publishedYes

Keywords

  • Classification
  • L penalty
  • Smoothing spline ANOVA
  • Sparsity
  • Support vector machine

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint Dive into the research topics of 'Variable selection for support vector machines via smoothing spline anova'. Together they form a unique fingerprint.

  • Cite this