Logo image
Back
Directorial Editing: A Hybrid Deep-Learning Approach to Content-Aware Image Retargeting and Resizing
Journal article   Open access   Peer reviewed

Directorial Editing: A Hybrid Deep-Learning Approach to Content-Aware Image Retargeting and Resizing

Elliot Dickman and Paul Diefenbach
Electronics (Basel), v 13(22), 4459
14 Nov 2024
DOI: https://doi.org/10.3390/electronics13224459
url
https://doi.org/10.3390/electronics13224459View
Published, Version of Record (VoR) Open

Abstract

computational photography artificial intelligence image composition
Image retargeting is a common computer graphics task which involves manipulating the size or aspect ratio of an image. This task often presents a challenge to the artist or user, because manipulating the size of an image necessitates some degree of data loss as pixels need to be removed to accommodate a different image size. We present an image retargeting framework which implements a confidence map generated by a segmentation model for content-aware resizing, allowing users to specify which subjects in an image to preserve using natural language prompts much like the role of an art director conversing with their artist. Using computer vision models to detect object positions also provides additional control over the composition of the retargeted image at various points in the image-processing pipeline. This object-based approach to energy map augmentation is incredibly flexible, because only minor adjustments to the processing of the energy maps can provide a significant degree of control over where seams—paths of pixels through the image—are removed, and how seam removal is prioritized in different sections of the image. It also provides additional control with techniques for object and background separation and recomposition. This research explores how several different types of deep-learning models can be integrated into this pipeline in order to easily make these decisions, and provide different retargeting results on the same image based on user input and compositional considerations. Because this is a framework based on existing machine-learning models, this approach will benefit from advancements in the rapidly developing fields of computer vision and large language models and can be extended for further natural language directorial controls over images.

Metrics

13 Record Views
1 citations in Web of Science

Details

InCites Highlights

Data related to this publication, from InCites Benchmarking & Analytics tool:

Web of Science research areas
Computer Science, Information Systems
Engineering, Electrical & Electronic
Physics, Applied
Logo image