Feature Selection for Ensembles

David W. Opitz, University of Montana

The traditional motivation behind feature selection algorithms is to find the best subset of features for a task using one particular learning algorithm. Given the recent success of ensembles, however, we investigate the notion of ensemble feature selection. This task is harder than traditional feature selection in that one not only needs to find features germane to the learning task and learning algorithm, but one also needs to find a set of feature subsets that will promote disagreement among the ensemble’s classifiers. In this paper, we present an ensemble feature selection approach that is based on genetic algorithms. Though conceptually simple, our algorithm shows improved performance over the popular and powerful ensemble approaches of Bagging and Ada-boosting and demonstrates the utility of ensemble feature selection.


This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.