Files and links (1)
SubmittedOpen Access (License Unspecified), Open
Conference proceeding
Do Deepfakes Feel Emotions? A Semantic Approach to Detecting Deepfakes Via Emotional Inconsistencies
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 1013-1022
Jun 2021
Abstract
Recent advances in deep learning and computer vision have spawned a new class of media forgeries known as deepfakes, which typically consist of artificially generated human faces or voices. The creation and distribution of deepfakes raise many legal and ethical concerns. As a result, the ability to distinguish between deepfakes and authentic media is vital. While deepfakes can create plausible video and audio, it may be challenging for them to to generate content that is consistent in terms of high-level semantic features, such as emotions. Unnatural displays of emotion, measured by features such as valence and arousal, can provide significant evidence that a video has been synthesized. In this paper, we propose a novel method for detecting deepfakes of a human speaker using the emotion predicted from the speaker's face and voice. The proposed technique leverages Long Short-Term Memory (LSTM) networks that predict emotion from audio and video Low-Level Descriptors (LLDs). Predicted emotion in time is used to classify videos as authentic or deepfakes through an additional supervised classifier.
Metrics
Details
- Title
- Do Deepfakes Feel Emotions? A Semantic Approach to Detecting Deepfakes Via Emotional Inconsistencies
- Creators
- Brian Hosler - Drexel UniversityDavide Salvi - Politecnico di MilanoAnthony Murray - Drexel UniversityFabio Antonacci - Politecnico di MilanoPaolo Bestagini - Politecnico di MilanoStefano Tubaro - Politecnico di MilanoMatthew C Stamm - Drexel UniversityIEEE Comp Soc
- Publication Details
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 1013-1022
- Publisher
- IEEE
- Grant note
- Arm (10.13039/100016311) Air Force Research Laboratory (10.13039/100006602) National Science Foundation (10.13039/100000001)
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Electrical and Computer Engineering
- Web of Science ID
- WOS:000705890201011
- Scopus ID
- 2-s2.0-85113722839
- Other Identifier
- 991019168669704721
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- International collaboration
- Web of Science research areas
- Computer Science, Artificial Intelligence