High Dimensional Non Linear Variable Selection through Hierarchical Kernel Learning

Francis Bach

Description

Niveau: Supérieur, Doctorat, Bac+8
High-Dimensional Non-Linear Variable Selection through Hierarchical Kernel Learning Francis Bach INRIA - WILLOW Project-Team Laboratoire d'Informatique de l'Ecole Normale Superieure (CNRS/ENS/INRIA UMR 8548) 23, avenue d'Italie, 75214 Paris, France September 4, 2009 Abstract We consider the problem of high-dimensional non-linear variable selection for supervised learning. Our approach is based on performing linear selection among exponentially many ap- propriately defined positive definite kernels that characterize non-linear interactions between the original variables. To select efficiently from these many kernels, we use the natural hierar- chical structure of the problem to extend the multiple kernel learning framework to kernels that can be embedded in a directed acyclic graph; we show that it is then possible to perform kernel selection through a graph-adapted sparsity-inducing norm, in polynomial time in the number of selected kernels. Moreover, we study the consistency of variable selection in high-dimensional settings, showing that under certain assumptions, our regularization framework allows a num- ber of irrelevant variables which is exponential in the number of observations. Our simulations on synthetic datasets and datasets from the UCI repository show state-of-the-art predictive per- formance for non-linear regression problems. 1 Introduction High-dimensional problems represent a recent and important topic in machine learning, statistics and signal processing.

• kernel-based methods


• linear variable


• methods


• selection


• multiple kernel


• generic kernel-based algorithms


• sparsity- inducing norms


• regression


• learning

Sujets

• Sélection
• Régression
• Learning

Informations

• Publié par: profil-zyak-2012
• Nombre de lectures: 8
• Langue: English

Extrait