Loading...
Please wait, while we are loading the content...
Utterance Level Feature Aggregation with Deep Metric Learning for Speech Emotion Recognition
| Content Provider | MDPI |
|---|---|
| Author | Mocanu, Bogdan Tapu, Ruxandra Zaharia, Titus |
| Copyright Year | 2021 |
| Description | Emotion is a form of high-level paralinguistic information that is intrinsically conveyed by human speech. Automatic speech emotion recognition is an essential challenge for various applications; including mental disease diagnosis; audio surveillance; human behavior understanding; e-learning and human–machine/robot interaction. In this paper, we introduce a novel speech emotion recognition method, based on the Squeeze and Excitation ResNet (SE-ResNet) model and fed with spectrogram inputs. In order to overcome the limitations of the state-of-the-art techniques, which fail in providing a robust feature representation at the utterance level, the CNN architecture is extended with a trainable discriminative GhostVLAD clustering layer that aggregates the audio features into compact, single-utterance vector representation. In addition, an end-to-end neural embedding approach is introduced, based on an emotionally constrained triplet loss function. The loss function integrates the relations between the various emotional patterns and thus improves the latent space data representation. The proposed methodology achieves 83.35% and 64.92% global accuracy rates on the RAVDESS and CREMA-D publicly available datasets, respectively. When compared with the results provided by human observers, the gains in global accuracy scores are superior to 24%. Finally, the objective comparative evaluation with state-of-the-art techniques demonstrates accuracy gains of more than 3%. |
| Starting Page | 4233 |
| e-ISSN | 14248220 |
| DOI | 10.3390/s21124233 |
| Journal | Sensors |
| Issue Number | 12 |
| Volume Number | 21 |
| Language | English |
| Publisher | MDPI |
| Publisher Date | 2021-06-20 |
| Access Restriction | Open |
| Subject Keyword | Sensors Artificial Intelligence Deep Convolution Neural Networks Speech Emotion Recognition Emotion Metric Learning Utterance Level Feature Aggregation |
| Content Type | Text |
| Resource Type | Article |
