Thomas Navin Lal, Olivier Chapelle, Jason Weston, André Elisseeff
Although many embedded feature selection methods have been introduced during the last few years, a unifying theoretical framework has not been developed to date. We start this chapter by defining...
Yoshua Bengio, Nicolas Chapados, Isabelle Guyon, André Elisseeff
Metric-based methods have recently been introduced for model selection and regularization, often yielding very significant improvements over the alternatives tried (including crossvalidation). All...
Regularization, Kernels and Sigmoid Net (2007)
Stéphane Canu, André Elisseeff, Bp Place, Emile Blondel, Mont Saint Aignan France
Many Neural networks use sigmoid--shaped functions because of biological inspired arguments. But since they work well there must be some mathematical reason for it. This work aims at presenting such...
Why Sigmoid-Shaped Functions Are Good for Learning? (2007)
Stéphane Canu, André Elisseeff, Gsi Psi, Bp Place, Emile Blondel, Eric Universit'e Lumi`ere, ...
ffl Des donn'ees : Sn = (X i ; Y i ) i=1;n , un 'echantillon i.i.d. tir'e suivant la distribution IP(x; y) inconnue. Stephane.Canu@insa-rouen.fr y Andre.Elisseeff@univ-lyon2.fr ffl Un...
Theodoros Evgeniou, Massimiliano Pontil, André Elisseeff
Leave one out error, stability, and generalization of
Overfitting in making comparisons between variable selection methods (2003)
Juha Reunanen, Isabelle Guyon, André Elisseeff
This paper addresses a common methodological flaw in the comparison of variable selection methods. A practical approach to guide the search or the selection process is to compute cross-validation...
Dimensionality Reduction via Sparse Support Vector Machines (2003)
Jinbo Bi, Kristin P. Bennett, Mark Embrechts, Curt M. Breneman, Minghu Song, Isabelle Guyon, ...
We describe a methodology for performing variable ranking and selection using support vector machines (SVMs). The method constructs a series of sparse linear SVMs to generate linear models that can...
Distributional word clusters vs. words for text categorization (2003)
Ron Bekkerman, Ran El-yaniv, Naftali Tishby, Yoad Winter, Isabelle Guyon, André Elisseeff
We study an approach to text categorization that combines distributional clustering of words and a Support Vector Machine (SVM) classifier. This word-cluster representation is computed using the...
Grafting: Fast, incremental feature selection by gradient descent in function space (2003)
Simon Perkins, Kevin Lacker, James Theiler, Isabelle Guyon, André Elisseeff
We present a novel and flexible approach to the problem of feature selection, called grafting. Rather than considering feature selection as separate from learning, grafting treats the selection of...
Grafting: Fast, incremental feature selection by gradient descent in function space (2003)
Simon Perkins, Kevin Lacker, James Theiler, Isabelle Guyon, André Elisseeff
We present a novel and flexible approach to the problem of feature selection, called grafting. Rather than considering feature selection as separate from learning, grafting treats the selection of...
Sufficient dimensionality reduction (2003)
Amir Globerson, Naftali Tishby, Isabelle Guyon, André Elisseeff
Dimensionality reduction of empirical co-occurrence data is a fundamental problem in unsupervised learning. It is also a well studied problem in statistics known as the analysis of cross-classified...
Feature extraction by non-parametric mutual information maximization (2003)
Kari Torkkola, Isabelle Guyon, André Elisseeff
We present a method for learning discriminative feature transforms using as criterion the mutual information between class labels and transformed features. Instead of a commonly used mutual...
A Divisive Information-Theoretic Feature Clustering Algorithm for Text Classification (2003)
Inderjit S. Dhillon, Subramanyam Mallela, Rahul Kumar, Isabelle Guyon, André Elisseeff
High dimensionality of text can be a deterrent in applying complex learners such as Support Vector Machines to the task of text classification. Feature clustering is a powerful alternative to feature...
An extensive empirical study of feature selection metrics for text classification (2003)
George Forman, Isabelle Guyon, André Elisseeff
Machine learning for text classification is the cornerstone of document categorization, news filtering, document routing, and personalization. In text domains, effective feature selection is...
Use of the zero-norm with linear models and kernel methods (2003)
Jason Weston, André Elisseeff, Bernhard Schölkopf, Pack Kaelbling
We explore the use of the so-called zero-norm of the parameters of linear models in learning. Minimization of such a quantity has many uses in a machine learning context: for variable or feature...
Dimensionality Reduction via Sparse Support Vector Machines (2003)
Jinbo Bi, Kristin P. Bennett, Mark Embrechts, Curt M. Breneman, Minghu Song, Isabelle Guyon, ...
We describe a methodology for performing variable ranking and selection using support vector machines (SVMs). The method constructs a series of sparse linear SVMs to generate linear models that can...
Rich Caruana, Virginia R. De Sa, Isabelle Guyon, André Elisseeff
In supervised learning variable selection is used to find a subset of the available inputs that accurately predict the output. This paper shows that some of the variables that variable selection...
An extensive empirical study of feature selection metrics for text classification (2003)
George Forman, Isabelle Guyon, André Elisseeff
Machine learning for text classification is the cornerstone of document categorization, news filtering, document routing, and personalization. In text domains, effective feature selection is...
Dimensionality Reduction via Sparse Support Vector Machines (2003)
Jinbo Bi, Kristin P. Bennett, Mark Embrechts, Curt M. Breneman, Minghu Song, Isabelle Guyon, ...
We describe a methodology for performing variable ranking and selection using support vector machines (SVMs). The method constructs a series of sparse linear SVMs to generate linear models that can...
An extensive empirical study of feature selection metrics for text classification (2003)
George Forman, Isabelle Guyon, André Elisseeff
Machine learning for text classification is the cornerstone of document categorization, news filtering, document routing, and personalization. In text domains, effective feature selection is...
Extensions to metric-based model selection (2003)
Yoshua Bengio, Nicolas Chapados, Isabelle Guyon, André Elisseeff
Metric-based methods have recently been introduced for model selection and regularization, often yielding very significant improvements over the alternatives tried (including cross-validation). All...
An extensive empirical study of feature selection metrics for text classification (2003)
George Forman, Isabelle Guyon, André Elisseeff
Machine learning for text classification is the cornerstone of document categorization, news filtering, document routing, and personalization. In text domains, effective feature selection is...
Variable Selection Using SVM-based Criteria (2003)
Alain Rakotomamonjy, Isabelle Guyon, André Elisseeff
We propose new methods to evaluate variable subset relevance with a view to variable selection.
Léon Personnaz, Isabelle Guyon, André Elisseeff
This paper presents a model selection procedure which stresses the importance of the classic polynomial models as tools for evaluating the complexity of a given modeling problem, and for removing...
Feature selection and transduction for prediction of molecular bioactivity for drug design (2003)
Weston, Jason, Pérez-Cruz, Fernando, Bousquet, Olivier, Chapelle, Olivier, Elisseeff, André, Schölkopf, Bernhard
Motivation: In drug discovery a key task is to identify characteristics that separate active (binding) compounds from inactive (non-binding) ones. An automated prediction system can help reduce...
Bound on the Risk for M-SVMs (2002)
Yann Guermeur, André Elisseeff, Dominique Zelus, Av Presidente, Juan D. Pern
Introduction From a theoretical point of view, the most appealing property of SVMs [12] is the fact that they represent straightforward implementations of Vapnik's structural risk minimization...
Stability and generalization (2002)
Olivier Bousquet, André Elisseeff, Dana Ron
We define notions of stability for learning algorithms and show how to use these notions to derive generalization error bounds based on the empirical error and the leave-one-out error. The methods we...
A kernel method for multi-labelled classification (2001)
This article presents a Support Vector Machine (SVM) like learning system to handle multi-label problems. Such problems are usually decomposed into many two-class problems but the expressive power of...
A kernel method for multi-labelled classification (2001)
This article presents a Support Vector Machine (SVM) like learning system to handle multi-label problems. Such problems are usually decomposed into many two-class problems but the expressive power of...
A Study About Algorithmic Stability and Its Relation to Generalization (2000)
This technical report presents some results about how to control the generalization error for stable algorithms. We define a new notion of stable algorithm and derive confidence bounds. It is shown...
Algorithmic Stability and Generalization Performance (2000)
Olivier Bousquet, André Elisseeff, F- Bron Cedex
We present a novel way of obtaining PAC-style bounds on the generalization error of learning algorithms, explicitly using their stability properties. A stable learner being one for which the learned...
Regularization, Kernels and Sigmoid Nets (1999)
Stéphane Canu, André Elisseeff, Bp Place, Emile Blondel, Mont Saint Aignan France
Many Neural networks use sigmoid--shaped functions because of biology-inspired arguments. But there must be some mathematical reason for their efficiency. This work aims at presenting such a...
Margin Error and Generalization Capabilities of Multi-Class Discriminant Systems (1999)
André Elisseeff, Yann Guermeur, Yann Guermeur Loria, Hélène Paugam-Moisy
The theory and practice of discriminant analysis have been mainly developed for two-class problems (computation of dichotomies). This phenomenon can easily be explained, since there is an obvious way...
JNN, a Randomized Algorithm for Learning Multilayer Networks in Polynomial Time (1998)
André Elisseeff, Hélène Paugam-Moisy, Ecole Normale, Suprieure Lyon
From an analytical approach of the multilayer architecture, we deduce a polynomialtime algorithm for learning from examples. We call it JNN, for "Jacobian Neural Network". Although this...
Subagging for credit scoring models
Paleologo, Giuseppe, Elisseeff, André, Antonini, Gianluca
The logistic regression framework has been for long time the most used statistical method when assessing customer credit risk. Recently, a more pragmatic approach has been adopted, where the first...