Files and links (1)
Published, Version of Record (VoR)CC BY V4.0, Open
Journal article
Fusion-ConvBERT: Parallel Convolution and BERT Fusion for Speech Emotion Recognition
Sensors (Basel, Switzerland), v 20(22), pp 1-19
23 Nov 2020
PMID: 33238396
Abstract
Speech emotion recognition predicts the emotional state of a speaker based on the person's speech. It brings an additional element for creating more natural human-computer interactions. Earlier studies on emotional recognition have been primarily based on handcrafted features and manual labels. With the advent of deep learning, there have been some efforts in applying the deep-network-based approach to the problem of emotion recognition. As deep learning automatically extracts salient features correlated to speaker emotion, it brings certain advantages over the handcrafted-feature-based methods. There are, however, some challenges in applying them to the emotion recognition problem, because data required for properly training deep networks are often lacking. Therefore, there is a need for a new deep-learning-based approach which can exploit available information from given speech signals to the maximum extent possible. Our proposed method, called "Fusion-ConvBERT", is a parallel fusion model consisting of bidirectional encoder representations from transformers and convolutional neural networks. Extensive experiments were conducted on the proposed model using the EMO-DB and Interactive Emotional Dyadic Motion Capture Database emotion corpus, and it was shown that the proposed method outperformed state-of-the-art techniques in most of the test configurations.
Metrics
Details
- Title
- Fusion-ConvBERT: Parallel Convolution and BERT Fusion for Speech Emotion Recognition
- Creators
- Sanghyun Lee - Department of Electronics and Electrical Engineering, Korea University, Seoul 136-713, KoreaDavid K Han - Drexel UniversityHanseok Ko - Korea University
- Publication Details
- Sensors (Basel, Switzerland), v 20(22), pp 1-19
- Publisher
- MDPI
- Grant note
- 2019R1A2C2009480 / National Research Foundation (NRF) grant funded by the MSIP of Korea
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- Electrical and Computer Engineering
- Web of Science ID
- WOS:000595199400001
- Scopus ID
- 2-s2.0-85096818718
- Other Identifier
- 991019168851504721
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- International collaboration
- Web of Science research areas
- Chemistry, Analytical
- Engineering, Electrical & Electronic
- Instruments & Instrumentation