Files and links (1)
Preprint (Author's original)arXiv.org - Non-exclusive license to distribute, Open
Preprint
Progressive Weight Pruning of Deep Neural Networks using ADMM
arXiv (Cornell University)
16 Oct 2018
Abstract
Deep neural networks (DNNs) although achieving human-level performance in
many domains, have very large model size that hinders their broader
applications on edge computing devices. Extensive research work have been
conducted on DNN model compression or pruning. However, most of the previous
work took heuristic approaches. This work proposes a progressive weight pruning
approach based on ADMM (Alternating Direction Method of Multipliers), a
powerful technique to deal with non-convex optimization problems with
potentially combinatorial constraints. Motivated by dynamic programming, the
proposed method reaches extremely high pruning rate by using partial prunings
with moderate pruning rates. Therefore, it resolves the accuracy degradation
and long convergence time problems when pursuing extremely high pruning ratios.
It achieves up to 34 times pruning rate for ImageNet dataset and 167 times
pruning rate for MNIST dataset, significantly higher than those reached by the
literature work. Under the same number of epochs, the proposed method also
achieves faster convergence and higher compression rates. The codes and pruned
DNN models are released in the link bit.ly/2zxdlss
Metrics
4 Record Views
Details
- Title
- Progressive Weight Pruning of Deep Neural Networks using ADMM
- Creators
- Shaokai YeTianyun ZhangKaiqi ZhangJiayu LiKaidi XuYunfei YangFuxun YuJian TangMakan FardadSijia LiuXiang ChenXue LinYanzhi Wang
- Publication Details
- arXiv (Cornell University)
- Resource Type
- Preprint
- Language
- English
- Academic Unit
- Computer Science (Computing)
- Other Identifier
- 991021871483904721