Ensembles of Budgeted Kernel Support Vector Machines for Parallel Large Scale Learning

Ensembles of Budgeted Kernel Support Vector Machines for Parallel Large Scale Learning

Lévesque, Julien-Charles and Gagné, Christian and Sabourin, Robert

NIPS 2013 Workshop on Big Learning: Advances in Algorithms and Data Management 2013

Abstract : In this work, we propose to combine multiple budgeted kernel support vector ma-chines (SVMs) trained with stochastic gradient descent (SGD) in order to exploit large databases and parallel computing resources. The variance induced by budget restrictions of the kernel SVMs is reduced through the averaging of predictions, resulting in greater generalization performance. The variance of the trainings re-sults in a diversity of predictions, which can help explain the better performance. Finally, the proposed method is intrinsically parallel, which means that parallel computing resources can be exploited in a straightforward manner.