Conference proceeding
Sparse Kernel Transfer Learning
ADVANCES IN VISUAL COMPUTING, ISVC 2022, PT II, v 13599, pp 39-52
01 Jan 2022
Featured in Collection : UN Sustainable Development Goals @ Drexel
Abstract
Deep learning performs remarkably well in a supervised learning setting. For computer vision tasks, the standard architecture is a convolutional neural network containing several layers of convolution and pooling, followed by a fully connected layer with a softmax output layer. Due to advancements in algorithms and activations, random initialization of the weights of the network can be effectively trained through stochastic gradient descent. In the event that you have a smaller dataset, one can utilize a transfer learning technique of the network weights in order to achieve state-of-the-art accuracy in over-parameterized models. We demonstrate that a transfer learning of a different kind yields better results over a randomized initialization of the network, and even over the standard transfer learning method in specific situations. In this paper, we describe sparse kernel transfer learning, a method that utilizes sparse coding and dictionary learning to pre-train the filters of a CNN. This pre-training is reminiscent of the unsupervised autoencoder training that used to be performed when stacking layers of a neural network. We argue that this dictionary transfer provides a better initialization, results in better classification, is more interpretable, and can even outperform the state-of-the-art models in transfer learning in certain datasets.
Metrics
7 Record Views
Details
- Title
- Sparse Kernel Transfer Learning
- Creators
- Edward Kim - Drexel UniversityTrang Ha - Drexel UniversityGarrett T. Kenyon - Los Alamos National Laboratory
- Contributors
- G Bebis (Editor)B Li (Editor)A Yao (Editor)Y Liu (Editor)Y Duan (Editor)M Lau (Editor)R Khadka (Editor)A Crisan (Editor)R Chang (Editor)
- Publication Details
- ADVANCES IN VISUAL COMPUTING, ISVC 2022, PT II, v 13599, pp 39-52
- Series
- Lecture Notes in Computer Science
- Publisher
- Springer Nature
- Number of pages
- 14
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Computer Science (Computing)
- Web of Science ID
- WOS:000916279000004
- Scopus ID
- 2-s2.0-85145252720
- Other Identifier
- 991021884690204721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- Web of Science research areas
- Computer Science, Software Engineering
- Computer Science, Theory & Methods