Journal article
Recognizing Material Properties from Images
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, v 42(8), pp 1981-1995
01 Aug 2020
PMID: 30932831
Featured in Collection : UN Sustainable Development Goals @ Drexel
Abstract
Humans implicitly rely on the properties of materials to guide our interactions. Grasping smooth materials, for example, requires more care than rough ones. We may even visually infer non-visual properties (e.g., softness is a physical material property). We refer to visually-recognizable material properties as visual material attributes. Recognizing these attributes in images can provide valuable information for scene understanding and material recognition. Unlike typical object and scene attributes, however, visual material attributes are local (i.e., fuzziness does not have a shape). Given full supervision, we may accurately recognize such attributes from purely local information (small image patches). Obtaining consistent full supervision at scale, however, is challenging. To solve this problem, we probe the human visual perception of materials. By asking simple yes/no questions comparing pairs of image patches, we obtain the weak supervision required to build a set of classifiers for attributes that, while unnamed, function similarly to the attributes with which we describe materials. Furthermore, we integrate this method in the end-to-end learning of a CNN that simultaneously recognizes materials and their visual attributes. Experiments show that visual material attributes serve as both a useful representation for known material categories and as a basis for transfer learning.
Metrics
Details
- Title
- Recognizing Material Properties from Images
- Publication Details
- IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, v 42(8), pp 1981-1995
- Publisher
- IEEE COMPUTER SOC; LOS ALAMITOS
- Number of pages
- 0
- Grant note
- This work was supported by the Office of Naval Research grants N00014-16-1-2158 (N00014-14-1-0316) and N00014-17-1-2406, and the US National Science Foundation awards IIS-1421094 and IIS-1715251. The Titan X used for part of this research was donated by the NVIDIA Corporation.
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- Drexel University
- Web of Science ID
- WOS:000545415400013
- Scopus ID
- 2-s2.0-85087469800
- Other Identifier
- 991021860768004721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Web of Science research areas
- Computer Science, Artificial Intelligence
- Engineering, Electrical & Electronic