Logo image
Back
Towards Explainable Visual Emotion Understanding
Conference proceeding

Towards Explainable Visual Emotion Understanding

Yue Zhang, Wanying Ding, Ran Xu and Xiaohua Hu
2021 IEEE International Conference on Big Data (Big Data), pp 1155-1162
15 Dec 2021
DOI: https://doi.org/10.1109/BigData52589.2021.9671452
Featured in Collection :   UN Sustainable Development Goals @ Drexel

Abstract

Analytical models Big Data explainability Medical services Multimedia Web sites Natural languages Safety visual emotion analysis Visualization
Emotion understanding from an image is an important computer vision research topic, but the reasoning behind the justification has been largely unexplored. We propose to utilize language, and more specifically, explanation of the judgement to increase the explainability for this task. We collect a dataset with image, emotion tag and explanation in natural language, and conduct analysis on the dataset to gain insights on the ambiguity of the human emotion perceptual. We examine baseline methods to predict emotion from image, explanation and general image description, and unifying both modalities. Our experiments shed lights on effects from different modalities and we also identify opportunities for future visual emotion categorization research based on the analysis. We release our dataset to advance future research.

Metrics

13 Record Views

Details

UN Sustainable Development Goals (SDGs)

This publication has contributed to the advancement of the following goals:

#3 Good Health and Well-Being

InCites Highlights

Data related to this publication, from InCites Benchmarking & Analytics tool:

Collaboration types
Industry collaboration
Domestic collaboration
Web of Science research areas
Computer Science, Artificial Intelligence
Computer Science, Information Systems
Computer Science, Theory & Methods
Logo image