Feature selection for ensembles using the multi-objective optimization approach
Feature selection for ensembles using the multi-objective optimization approach
Oliveira, Luiz S. and Morita, Marisa and Sabourin, Robert
Studies in Computational Intelligence 2006
Abstract : Feature selection for ensembles has shown to be an effective strategy for ensemble creation due to its ability of producing good subsets of features, which make the classifiers of the ensemble disagree on dificult cases. In this paper we present an ensemble feature selection approach based on a hierarchical multi-objective geneticalgorithm. The underpinning paradigm is the “over produce and choose”. The algorithm operates in two levels. Firstly, it performs feature selection in order togenerate a set of classifiers and then it chooses the best team of classifiers. In order to show its robustness, the method is evaluated in two different contexts: supervised and unsupervised feature selection. In the former, we have considered the problem of handwritten digit recognition and used three different feature sets and multi-layer perceptron neural networks as classifiers. In the latter, we took into account the problem of handwritten month word recognition and used three different feature sets and hidden Markov models as classifiers. Experiments and comparisons with classical methods, such as Bagging and Boosting, demonstrated that the proposed methodology brings compelling improvements when classifiers have to work with very low error rates. © 2006 Springer-Verlag Berlin Heidelberg.
