Speech Emotion Recognition using Enriched Spectrogram and Deep Convolutional Neural Network Transfer Learning

Speech emotion recognition (SER) is a challenging field of research that has attracted attention during the last two decades. Feature extraction has been reported as the most challenging issue in SER systems. Deep neural networks could partially solve this problem in some other applications. In order to address this problem, we proposed a novel enriched spectrogram calculated based on the fusion of wide-band and narrow-band spectrograms. The proposed spectrogram benefited from both high temporal and spectral resolution. Then we applied the resultant spectrogram images to the pre-trained deep convolutional neural network, ResNet۱۵۲. Instead of the last layer of ResNet۱۵۲, we added five additional layers to adopt the model to the present task. All the experiments performed on the popular EmoDB dataset are based on leaving one speaker out of a technique that guarantees the speaker's independency from the model. The model gains an accuracy rate of ۸۸.۹۷% which shows the efficiency of the proposed approach in contrast to other state-of-the-art methods.

Keywords:

Wideband and narrowband spectrogram , ResNet۱۵۲ , DCNN , Transfer learning , Speech emotion recognition