Deep learning techniques for multimedia forensics

Belhassen Bayar

doi:10.17918/1c8b-m013

Digital images play an important role in a wide variety of settings such as news reporting, criminal investigations, etc. As a result, it is necessary to devise forensic algorithms capable of determining the source camera and the processing history of digital images. This is possible because both image editing operations and an image's source camera leave behind unique statistical traces in the same way that a criminal leaves behind fingerprints at a crime scene. Previous forensic techniques identify these fingerprints through theoretical analysis or develop heuristic features to capture their effects. These methods have their drawbacks where theoretical analysis is not always achievable and heuristic based approaches are often suboptimal. Recently, convolutional neural networks (CNNs) have gained significant attention due to their ability to adaptively learn classification features directly from data. While a forensic analyst can use CNNs to learn forensic fingerprints, this is problematic because CNNs in their standard form tend to learn features related to an image's content as opposed to learn forensic fingerprints which are content-independent. Thus, a forensic analyst must adapt the CNN to capture forensic fingerprints from images. To do this, they must develop new CNN architectures for different forensic tasks such as camera model identification and image editing detection. Additionally, CNNs in forensics must be robust to common post-processing operations which significantly deteriorate the performance of the existing forensic algorithms. Unfortunately, techniques from computer vision to increase CNN's robustness are not effective when designing and training forensic CNNs. As a result, new techniques must be developed to design and train forensic CNNs such that they are robust enough to operate in realistic scenarios. In this dissertation, we propose a set of new deep learning approaches to perform several multimedia forensic tasks. We first develop a new type of convolutional layer within a CNN, called a 'constrained convolutional layer', that can jointly suppress an image's content and adaptively learn low-level forensic classification features directly from data. The constrained CNN has proven effective at performing image manipulation detection, order of processing operations detection, image manipulation parameter estimation, and camera model feature extraction for unknown classes detection. Next, we propose the architectural design guidelines to build a new CNN architecture associated with a forensic data augmentation based training method that can capture camera's traces in post-processed color images. We demonstrate the advantage of our approach over the existing forensic methods at identifying camera models in post-processed color images through rigorous experiments and analysis.

Deep learning techniques for multimedia forensics

Files and links (1)

Abstract

Metrics

Details

Deep learning techniques for multimedia forensics

Files and links (1)

Abstract

Metrics

Details

Drexel University Social media