Sparse, Robust and Discriminative Representation by Supervised Regularized Auto-encoder

Recent researches have determined that regularized auto-encoders can provide a good representation of data which improves the performance of data classification. These type of auto-encoders which are usually over-complete, provide a representation of data that has some degree of sparsity and is robust against variation of data to extract meaningful information and reveal the underlying structure of data by making a change in classic auto- encoders’ structure and/or adding regularizing terms to the objective function. The present study aimed to propose a novel approach to generate sparse, robust, and discriminative features through supervised regularized auto-encoders, in which unlike most existing auto-encoders, the data labels are used during feature extraction to improve discrimination of the representation and also, the sparsity ratio of the representation is completely adaptive and dynamically determined based on data distribution and complexity. Results reveal that this method has better performance in comparison to other regularized auto-encoders regarding data classification. 

Keywords:

Supervised Auto-encoder , Feature Learning , Discriminative Representation , Manifold