Surface estimation, variable selection, and the nonparametric oracle property

Curtis B. Storlie, Howard D. Bondell, Brian J. Reich, Hao Zhang

Research output: Contribution to journalArticle

59 Citations (Scopus)

Abstract

Variable selection for multivariate nonparametric regression is an important, yet challenging, problem due, in part, to the infinite dimensionality of the function space. An ideal selection procedure would be automatic, stable, easy to use, and have desirable asymptotic properties. In particular, we define a selection procedure to be nonparametric oracle (np-oracle) if it consistently selects the correct subset of predictors and, at the same time, estimates the smooth surface at the optimal nonparametric rate, as the sample size goes to infinity. In this paper, we propose a model selection procedure for nonparametric models, and explore the conditions under which the new method enjoys the aforementioned properties. Developed in the framework of smoothing spline ANOVA, our estimator is obtained via solving a regularization problem with a novel adaptive penalty on the sum of functional component norms. Theoretical properties of the new estimator are established. Additionally, numerous simulations and examples suggest that the new approach substantially outperforms other existing methods in the finite sample setting.

Original languageEnglish (US)
Pages (from-to)679-705
Number of pages27
JournalStatistica Sinica
Volume21
Issue number2
StatePublished - Apr 2011
Externally publishedYes

Fingerprint

Oracle Property
Selection Procedures
Variable Selection
Estimator
Smoothing Splines
Multivariate Regression
Nonparametric Model
Smooth surface
Nonparametric Regression
Model Selection
Function Space
Asymptotic Properties
Dimensionality
Penalty
Predictors
Regularization
Sample Size
Infinity
Norm
Subset

Keywords

  • Adaptive LASSO
  • Nonparametric regression
  • Regularization method
  • Smoothing spline ANOVA
  • Variable selection

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Storlie, C. B., Bondell, H. D., Reich, B. J., & Zhang, H. (2011). Surface estimation, variable selection, and the nonparametric oracle property. Statistica Sinica, 21(2), 679-705.

Surface estimation, variable selection, and the nonparametric oracle property. / Storlie, Curtis B.; Bondell, Howard D.; Reich, Brian J.; Zhang, Hao.

In: Statistica Sinica, Vol. 21, No. 2, 04.2011, p. 679-705.

Research output: Contribution to journalArticle

Storlie, CB, Bondell, HD, Reich, BJ & Zhang, H 2011, 'Surface estimation, variable selection, and the nonparametric oracle property', Statistica Sinica, vol. 21, no. 2, pp. 679-705.
Storlie, Curtis B. ; Bondell, Howard D. ; Reich, Brian J. ; Zhang, Hao. / Surface estimation, variable selection, and the nonparametric oracle property. In: Statistica Sinica. 2011 ; Vol. 21, No. 2. pp. 679-705.
@article{2ec6c82a3f5f4529bdacc0884c7e1430,
title = "Surface estimation, variable selection, and the nonparametric oracle property",
abstract = "Variable selection for multivariate nonparametric regression is an important, yet challenging, problem due, in part, to the infinite dimensionality of the function space. An ideal selection procedure would be automatic, stable, easy to use, and have desirable asymptotic properties. In particular, we define a selection procedure to be nonparametric oracle (np-oracle) if it consistently selects the correct subset of predictors and, at the same time, estimates the smooth surface at the optimal nonparametric rate, as the sample size goes to infinity. In this paper, we propose a model selection procedure for nonparametric models, and explore the conditions under which the new method enjoys the aforementioned properties. Developed in the framework of smoothing spline ANOVA, our estimator is obtained via solving a regularization problem with a novel adaptive penalty on the sum of functional component norms. Theoretical properties of the new estimator are established. Additionally, numerous simulations and examples suggest that the new approach substantially outperforms other existing methods in the finite sample setting.",
keywords = "Adaptive LASSO, Nonparametric regression, Regularization method, Smoothing spline ANOVA, Variable selection",
author = "Storlie, {Curtis B.} and Bondell, {Howard D.} and Reich, {Brian J.} and Hao Zhang",
year = "2011",
month = "4",
language = "English (US)",
volume = "21",
pages = "679--705",
journal = "Statistica Sinica",
issn = "1017-0405",
publisher = "Institute of Statistical Science",
number = "2",

}

TY - JOUR

T1 - Surface estimation, variable selection, and the nonparametric oracle property

AU - Storlie, Curtis B.

AU - Bondell, Howard D.

AU - Reich, Brian J.

AU - Zhang, Hao

PY - 2011/4

Y1 - 2011/4

N2 - Variable selection for multivariate nonparametric regression is an important, yet challenging, problem due, in part, to the infinite dimensionality of the function space. An ideal selection procedure would be automatic, stable, easy to use, and have desirable asymptotic properties. In particular, we define a selection procedure to be nonparametric oracle (np-oracle) if it consistently selects the correct subset of predictors and, at the same time, estimates the smooth surface at the optimal nonparametric rate, as the sample size goes to infinity. In this paper, we propose a model selection procedure for nonparametric models, and explore the conditions under which the new method enjoys the aforementioned properties. Developed in the framework of smoothing spline ANOVA, our estimator is obtained via solving a regularization problem with a novel adaptive penalty on the sum of functional component norms. Theoretical properties of the new estimator are established. Additionally, numerous simulations and examples suggest that the new approach substantially outperforms other existing methods in the finite sample setting.

AB - Variable selection for multivariate nonparametric regression is an important, yet challenging, problem due, in part, to the infinite dimensionality of the function space. An ideal selection procedure would be automatic, stable, easy to use, and have desirable asymptotic properties. In particular, we define a selection procedure to be nonparametric oracle (np-oracle) if it consistently selects the correct subset of predictors and, at the same time, estimates the smooth surface at the optimal nonparametric rate, as the sample size goes to infinity. In this paper, we propose a model selection procedure for nonparametric models, and explore the conditions under which the new method enjoys the aforementioned properties. Developed in the framework of smoothing spline ANOVA, our estimator is obtained via solving a regularization problem with a novel adaptive penalty on the sum of functional component norms. Theoretical properties of the new estimator are established. Additionally, numerous simulations and examples suggest that the new approach substantially outperforms other existing methods in the finite sample setting.

KW - Adaptive LASSO

KW - Nonparametric regression

KW - Regularization method

KW - Smoothing spline ANOVA

KW - Variable selection

UR - http://www.scopus.com/inward/record.url?scp=78650732213&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78650732213&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:78650732213

VL - 21

SP - 679

EP - 705

JO - Statistica Sinica

JF - Statistica Sinica

SN - 1017-0405

IS - 2

ER -