Files and links (1)
PDFOpen Access (License Unspecified), Open Access
Thesis
Detecting deepfakes using emotional irregularities
Master of Science (M.S.), Drexel University
Jun 2020
DOI:
https://doi.org/10.17918/00000162
Abstract
Recent advances in deep learning have spawned a new class of media forgeries known as deepfakes, which typically consist of artificially generated human faces or voices. The creation and distribution of deepfakes raises many legal and ethical concerns. As a result, the ability to distinguish between deepfakes and authentic media is vital. Since deepfake generation methods are numerous and rapidly evolving, it is unrealistic to hope that a single method will be able to detect all deepfakes. Rather, a multiplicity of detection methods is needed, so that the burden of creating an undetectable deepfake becomes extremely high. While deepfakes can create plausible video and audio, it may be challenging for them to accurately synthesize high-level concepts such as emotion. Unnatural displays of emotion, as measured by valence and arousal, can provide important evidence that a video has been synthesized. In this work, we propose a novel method for detecting deepfakes of a human speaker using the emotion predicted from the face and voice. First, we train long short-term memory (LSTM) networks to predict emotion from low-level descriptors (LLDs) of audio and video of human speakers from the SEMAINE database. Then, we use the trained networks as feature extractors to extract sequences of valence and arousal from videos in a subset of the Deepfake Detection Challenge (DFDC) dataset. Finally, we classify the videos as authentic videos or deepfakes with high accuracy using features derived from the predicted emotion.
Metrics
60 File views/ downloads
104 Record Views
Details
- Title
- Detecting deepfakes using emotional irregularities
- Creators
- Anthony Francis Murray
- Contributors
- Matthew C. Stamm (Advisor)
- Awarding Institution
- Drexel University
- Degree Awarded
- Master of Science (M.S.)
- Publisher
- Drexel University; Philadelphia, Pennsylvania
- Number of pages
- ix, 46 pages
- Resource Type
- Thesis
- Language
- English
- Academic Unit
- College of Engineering (1970-2026); Electrical (and Computer) Engineering [Historical]; Drexel University
- Other Identifier
- 991014695232404721