Conference proceeding
Distributional Semantics of Line Charts for Trend Classification
ADVANCES IN VISUAL COMPUTING, ISVC 2022, PT II, v 13599, pp 259-269
01 Jan 2022
Abstract
Line charts are often used to convey high level information about time series data. Unfortunately, these charts are not always described in text, and as a result are often inaccessible to users with visual impairments who rely on screen readers. In these situations, an automated system that can describe the overall trend in a chart would be desirable. This paper presents a novel approach to classifying trends in line chart images, for use in existing chart summarization tools. Previous projects have introduced approaches to automatically summarize line charts, but have thus far been unable to describe chart trends with sufficient accuracy for real-world applications. Instead of classifying an image's trend via a convolutional neural network (CNN) system, as has been done previously, we present an architecture similar to bag-of-words (BoW) techniques for computer vision, mapping the image classification problem to an analogous natural language problem. We divided images into matrices of image patches which we then each treated as a series of "visual words" which were used to classify each image. We utilized natural language processing (NLP) word embeddings techniques to to create embeddings of visual words that allowed us to model contextual similarity between patches. We trained a linear support vector machine (SVM) model using these patch embeddings as inputs to classify the chart trend. We compared this method against a ResNet classifier pre-trained on ImageNet. Our experimental results showed that the novel approach presented in this paper outperforms existing approaches.
Metrics
4 Record Views
Details
- Title
- Distributional Semantics of Line Charts for Trend Classification
- Creators
- Connor Onweller - University of DelawareAndrew O'Brien - Drexel Univ, Philadelphia, PA 19104 USAEdward Kim - Drexel UniversityKathleen F. McCoy - University of Delaware
- Contributors
- G Bebis (Editor)B Li (Editor)A Yao (Editor)Y Liu (Editor)Y Duan (Editor)M Lau (Editor)R Khadka (Editor)A Crisan (Editor)R Chang (Editor)
- Publication Details
- ADVANCES IN VISUAL COMPUTING, ISVC 2022, PT II, v 13599, pp 259-269
- Series
- Lecture Notes in Computer Science
- Publisher
- Springer Nature
- Number of pages
- 11
- Grant note
- 1954364 / National Science Foundation; National Science Foundation (NSF) 1954364 / Direct For Computer & Info Scie & Enginr; Div Of Information & Intelligent Systems; National Science Foundation (NSF); NSF - Directorate for Computer & Information Science & Engineering (CISE)
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Computer Science (Computing)
- Web of Science ID
- WOS:000916279000020
- Scopus ID
- 2-s2.0-85145263891
- Other Identifier
- 991021884588204721
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- Web of Science research areas
- Computer Science, Software Engineering
- Computer Science, Theory & Methods