Variable selection for support vector machines via smoothing spline anova

Research output: Contribution to journalArticle

26 Citations (Scopus)

Abstract

It is well-known that the support vector machine paradigm is equivalent to solving a regularization problem in a reproducing kernel Hubert space. The squared norm penalty in the standard support vector machine controls the smoothness of the classification function. We propose, under the framework of smoothing spline ANOVA models, a new type of regularization to conduct simultaneous classification and variable selection in the SVM. The penalty functional used is the sum of functional component norms, which automatically applies soft-thresholding operations to functional components, hence yields sparse solutions. We suggest an efficient algorithm to solve the proposed optimization problem by iteratively solving quadratic and linear programming problems. Numerical studies, on both simulated data and real datasets, show that the modified support vector machine gives very competitive performances compared to other popular classification algorithms, in terms of both classification accuracy and variable selection.

Original languageEnglish (US)
Pages (from-to)659-674
Number of pages16
JournalStatistica Sinica
Volume16
Issue number2
StatePublished - Apr 2006
Externally publishedYes

Fingerprint

Smoothing Splines
Variable Selection
Support Vector Machine
Penalty
Regularization
Reproducing Kernel Space
Norm
Hubert Space
Thresholding
Classification Algorithm
Quadratic Programming
Linear programming
Numerical Study
Smoothness
Efficient Algorithms
Paradigm
Optimization Problem
Analysis of variance
Variable selection
Smoothing splines

Keywords

  • Classification
  • L penalty
  • Smoothing spline ANOVA
  • Sparsity
  • Support vector machine

ASJC Scopus subject areas

  • Mathematics(all)
  • Statistics and Probability

Cite this

Variable selection for support vector machines via smoothing spline anova. / Zhang, Hao.

In: Statistica Sinica, Vol. 16, No. 2, 04.2006, p. 659-674.

Research output: Contribution to journalArticle

@article{824d9cef39ce436a93b314d309c684fe,
title = "Variable selection for support vector machines via smoothing spline anova",
abstract = "It is well-known that the support vector machine paradigm is equivalent to solving a regularization problem in a reproducing kernel Hubert space. The squared norm penalty in the standard support vector machine controls the smoothness of the classification function. We propose, under the framework of smoothing spline ANOVA models, a new type of regularization to conduct simultaneous classification and variable selection in the SVM. The penalty functional used is the sum of functional component norms, which automatically applies soft-thresholding operations to functional components, hence yields sparse solutions. We suggest an efficient algorithm to solve the proposed optimization problem by iteratively solving quadratic and linear programming problems. Numerical studies, on both simulated data and real datasets, show that the modified support vector machine gives very competitive performances compared to other popular classification algorithms, in terms of both classification accuracy and variable selection.",
keywords = "Classification, L penalty, Smoothing spline ANOVA, Sparsity, Support vector machine",
author = "Hao Zhang",
year = "2006",
month = "4",
language = "English (US)",
volume = "16",
pages = "659--674",
journal = "Statistica Sinica",
issn = "1017-0405",
publisher = "Institute of Statistical Science",
number = "2",

}

TY - JOUR

T1 - Variable selection for support vector machines via smoothing spline anova

AU - Zhang, Hao

PY - 2006/4

Y1 - 2006/4

N2 - It is well-known that the support vector machine paradigm is equivalent to solving a regularization problem in a reproducing kernel Hubert space. The squared norm penalty in the standard support vector machine controls the smoothness of the classification function. We propose, under the framework of smoothing spline ANOVA models, a new type of regularization to conduct simultaneous classification and variable selection in the SVM. The penalty functional used is the sum of functional component norms, which automatically applies soft-thresholding operations to functional components, hence yields sparse solutions. We suggest an efficient algorithm to solve the proposed optimization problem by iteratively solving quadratic and linear programming problems. Numerical studies, on both simulated data and real datasets, show that the modified support vector machine gives very competitive performances compared to other popular classification algorithms, in terms of both classification accuracy and variable selection.

AB - It is well-known that the support vector machine paradigm is equivalent to solving a regularization problem in a reproducing kernel Hubert space. The squared norm penalty in the standard support vector machine controls the smoothness of the classification function. We propose, under the framework of smoothing spline ANOVA models, a new type of regularization to conduct simultaneous classification and variable selection in the SVM. The penalty functional used is the sum of functional component norms, which automatically applies soft-thresholding operations to functional components, hence yields sparse solutions. We suggest an efficient algorithm to solve the proposed optimization problem by iteratively solving quadratic and linear programming problems. Numerical studies, on both simulated data and real datasets, show that the modified support vector machine gives very competitive performances compared to other popular classification algorithms, in terms of both classification accuracy and variable selection.

KW - Classification

KW - L penalty

KW - Smoothing spline ANOVA

KW - Sparsity

KW - Support vector machine

UR - http://www.scopus.com/inward/record.url?scp=33746149382&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33746149382&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:33746149382

VL - 16

SP - 659

EP - 674

JO - Statistica Sinica

JF - Statistica Sinica

SN - 1017-0405

IS - 2

ER -