Book chapter
Learning Object Placement by Inpainting for Compositional Data Augmentation
Computer Vision – ECCV 2020, pp 566-581
28 Nov 2020
Abstract
We study the problem of common sense placement of visual objects in an image. This involves multiple aspects of visual recognition: the instance segmentation of the scene, 3D layout, and common knowledge of how objects are placed and where objects are moving in the 3D scene. This seemingly simple task is difficult for current learning-based approaches because of the lack of labeled training pair of foreground objects paired with cleaned background scenes. We propose a self-learning framework that automatically generates the necessary training data without any manual labeling by detecting, cutting, and inpainting objects from an image. We propose a PlaceNet that predicts a diverse distribution of common sense locations when given a foreground object and a background scene. We show one practical use of our object placement network for augmenting training datasets by recomposition of object-scene with a key property of contextual relationship preservation. We demonstrate improvement of object detection and instance segmentation performance on both Cityscape
[4] and KITTI
[9] datasets. We also show that the learned representation of our PlaceNet displays strong discriminative power in image retrieval and classification.
Metrics
2 Record Views
43 citations in Scopus
Details
- Title
- Learning Object Placement by Inpainting for Compositional Data Augmentation
- Creators
- Lingzhi Zhang - Philadelphia UniversityTarmily Wen - Philadelphia UniversityJie Min - Philadelphia UniversityJiancong Wang - Philadelphia UniversityDavid Han - DEVCOM Army Research LaboratoryJianbo Shi - Philadelphia University
- Contributors
- Andrea Vedaldi (Editor)Horst Bischof (Editor)Thomas Brox (Editor)Jan-Michael Frahm (Editor)
- Publication Details
- Computer Vision – ECCV 2020, pp 566-581
- Series
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing; Cham
- Number of pages
- 16
- Resource Type
- Book chapter
- Language
- English
- Academic Unit
- Electrical and Computer Engineering
- Scopus ID
- 2-s2.0-85097592640
- Other Identifier
- 991021930832504721