Logo image
Back
VideoFACT: Detecting Video Forgeries Using Attention, Scene Context, and Forensic Traces
Conference proceeding   Open access

VideoFACT: Detecting Video Forgeries Using Attention, Scene Context, and Forensic Traces

Tai D. Nguyen, Shengbang Fang and Matthew C. Stamm
2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp 8548-8558
03 Jan 2024
DOI: https://doi.org/10.1109/WACV57701.2024.00837
url
https://arxiv.org/pdf/2211.15775View
Preprint (Author's original)arXiv.org - Non-exclusive license to distribute,  Open

Abstract

Applications Codes Forensics Forgery Image forensics Social good Training Video coding Computer Vision
Fake videos represent an important misinformation threat. While existing forensic networks have demonstrated strong performance on image forgeries, recent results reported on the Adobe VideoSham dataset show that these networks fail to identify fake content in videos. In response, we propose VideoFACT - a new network that is able to detect and localize a wide variety of video forgeries and manipulations. To overcome challenges that existing networks face when analyzing videos, our network utilizes both forensic embeddings to capture traces left by manipulation, context embeddings to control for variation in forensic traces introduced by video coding, and a deep self-attention mechanism to estimate the quality and relative importance of local forensic embeddings. We create several new video forgery datasets and use these, along with publicly available data, to experimentally evaluate our network's performance. These results show that our proposed network is able to identify a diverse set of video forgeries, including those not encountered during training. Furthermore, we show that our network can be fine-tuned to achieve even stronger performance on challenging AI-based manipulations. (Code is available at: https://github.com/ductai199x/videofact-wacv-2024)

Metrics

10 Record Views
5 citations in Scopus

Details

Logo image