Extensions to Online Feature Selection Using Bagging and Boosting

Gregory Ditzler, Joseph Labarck, James Ritchie, Gail Rosen, Robi Polikar

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Feature subset selection can be used to sieve through large volumes of data and discover the most informative subset of variables for a particular learning problem. Yet, due to memory and other resource constraints (e.g., CPU availability), many of the state-of-the-art feature subset selection methods cannot be extended to high dimensional data, or data sets with an extremely large volume of instances. In this brief, we extend online feature selection (OFS), a recently introduced approach that uses partial feature information, by developing an ensemble of online linear models to make predictions. The OFS approach employs a linear model as the base classifier, which allows the l0-norm of the parameter vector to be constrained to perform feature selection leading to sparse linear models. We demonstrate that the proposed ensemble model typically yields a smaller error rate than any single linear model, while maintaining the same level of sparsity and complexity at the time of testing.

Original languageEnglish (US)
Article number8065078
Pages (from-to)4504-4509
Number of pages6
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume29
Issue number9
DOIs
StatePublished - Sep 2018

Fingerprint

Clustering algorithms
Feature extraction
Sieves
Program processors
Classifiers
Availability
Data storage equipment
Testing

Keywords

  • Ensembles
  • feature selection
  • online learning

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this

Extensions to Online Feature Selection Using Bagging and Boosting. / Ditzler, Gregory; Labarck, Joseph; Ritchie, James; Rosen, Gail; Polikar, Robi.

In: IEEE Transactions on Neural Networks and Learning Systems, Vol. 29, No. 9, 8065078, 09.2018, p. 4504-4509.

Research output: Contribution to journalArticle

Ditzler, Gregory ; Labarck, Joseph ; Ritchie, James ; Rosen, Gail ; Polikar, Robi. / Extensions to Online Feature Selection Using Bagging and Boosting. In: IEEE Transactions on Neural Networks and Learning Systems. 2018 ; Vol. 29, No. 9. pp. 4504-4509.
@article{b9cb79f8c62a498bb87cc2d50cd89ec6,
title = "Extensions to Online Feature Selection Using Bagging and Boosting",
abstract = "Feature subset selection can be used to sieve through large volumes of data and discover the most informative subset of variables for a particular learning problem. Yet, due to memory and other resource constraints (e.g., CPU availability), many of the state-of-the-art feature subset selection methods cannot be extended to high dimensional data, or data sets with an extremely large volume of instances. In this brief, we extend online feature selection (OFS), a recently introduced approach that uses partial feature information, by developing an ensemble of online linear models to make predictions. The OFS approach employs a linear model as the base classifier, which allows the l0-norm of the parameter vector to be constrained to perform feature selection leading to sparse linear models. We demonstrate that the proposed ensemble model typically yields a smaller error rate than any single linear model, while maintaining the same level of sparsity and complexity at the time of testing.",
keywords = "Ensembles, feature selection, online learning",
author = "Gregory Ditzler and Joseph Labarck and James Ritchie and Gail Rosen and Robi Polikar",
year = "2018",
month = "9",
doi = "10.1109/TNNLS.2017.2746107",
language = "English (US)",
volume = "29",
pages = "4504--4509",
journal = "IEEE Transactions on Neural Networks and Learning Systems",
issn = "2162-237X",
publisher = "IEEE Computational Intelligence Society",
number = "9",

}

TY - JOUR

T1 - Extensions to Online Feature Selection Using Bagging and Boosting

AU - Ditzler, Gregory

AU - Labarck, Joseph

AU - Ritchie, James

AU - Rosen, Gail

AU - Polikar, Robi

PY - 2018/9

Y1 - 2018/9

N2 - Feature subset selection can be used to sieve through large volumes of data and discover the most informative subset of variables for a particular learning problem. Yet, due to memory and other resource constraints (e.g., CPU availability), many of the state-of-the-art feature subset selection methods cannot be extended to high dimensional data, or data sets with an extremely large volume of instances. In this brief, we extend online feature selection (OFS), a recently introduced approach that uses partial feature information, by developing an ensemble of online linear models to make predictions. The OFS approach employs a linear model as the base classifier, which allows the l0-norm of the parameter vector to be constrained to perform feature selection leading to sparse linear models. We demonstrate that the proposed ensemble model typically yields a smaller error rate than any single linear model, while maintaining the same level of sparsity and complexity at the time of testing.

AB - Feature subset selection can be used to sieve through large volumes of data and discover the most informative subset of variables for a particular learning problem. Yet, due to memory and other resource constraints (e.g., CPU availability), many of the state-of-the-art feature subset selection methods cannot be extended to high dimensional data, or data sets with an extremely large volume of instances. In this brief, we extend online feature selection (OFS), a recently introduced approach that uses partial feature information, by developing an ensemble of online linear models to make predictions. The OFS approach employs a linear model as the base classifier, which allows the l0-norm of the parameter vector to be constrained to perform feature selection leading to sparse linear models. We demonstrate that the proposed ensemble model typically yields a smaller error rate than any single linear model, while maintaining the same level of sparsity and complexity at the time of testing.

KW - Ensembles

KW - feature selection

KW - online learning

UR - http://www.scopus.com/inward/record.url?scp=85031778862&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85031778862&partnerID=8YFLogxK

U2 - 10.1109/TNNLS.2017.2746107

DO - 10.1109/TNNLS.2017.2746107

M3 - Article

C2 - 29028210

AN - SCOPUS:85031778862

VL - 29

SP - 4504

EP - 4509

JO - IEEE Transactions on Neural Networks and Learning Systems

JF - IEEE Transactions on Neural Networks and Learning Systems

SN - 2162-237X

IS - 9

M1 - 8065078

ER -