An empirical evaluation of bagging and boosting for artificial neural networks

Bagging and boosting are two relatively new but popular methods for producing classifier ensembles. An ensemble consists a set of independently trained classifiers (such as neural networks or decision trees) whose predictions are combined when classifying instances. Previous research suggests that an ensemble as a whole is often more accurate than any of the single classifiers in the ensemble. In this paper we evaluate bagging and boosting as methods for creating an ensemble of neural networks. We also include results from Quinlan's (1996) decision tree evaluation of these methods. Our results indicate that the ensemble methods can indeed produce very accurate classifiers for some dataset, but that these gains may depend on aspects of the dataset. In particular we find that bagging is probably appropriate for most problems, but when properly applied boosting may produce even larger gains in accuracy.