Transformation-based Regularization for Information-based Deep Clustering

Transformation-based Regularization for Information-based Deep Clustering

Peng, Jizong and Desrosiers, Christian and Pedersoli, Marco

arXiv 2019

Abstract : The joint learning of a feature representation and a clustering of the training data is a powerful approach for unsupervised learning in which data is automatically divided in clusters that represent semantic classes. Recent clustering approaches based on mutual information maximization have achieved excellent results, sometimes comparable with fully-supervised approaches. In this work, we present a generalization of information-based deep clustering where two key factors are evaluated: i) the variables for which we want to maximize the mutual information, ii) the regularization of the mutual information loss by the use of image transformations. Through an extensive analysis, we show that maximizing the mutual information between a sample and its transformed version, with an additional regularization to make the learning smoother, outperforms previous approaches and leads to state of the art results on three different datasets. Additional experiments show that the proposed method largely outperforms disentangling methods for classification tasks and is useful as unsupervised initialization for supervised learning.