Files and links (1)
Preprint (Author's original)arXiv.org - Non-exclusive license to distribute, Open
Preprint
Pursing the Sparse Limitation of Spiking Deep Learning Structures
arXiv.org
18 Nov 2023
Abstract
Spiking Neural Networks (SNNs), a novel brain-inspired algorithm, are
garnering increased attention for their superior computation and energy
efficiency over traditional artificial neural networks (ANNs). To facilitate
deployment on memory-constrained devices, numerous studies have explored SNN
pruning. However, these efforts are hindered by challenges such as scalability
challenges in more complex architectures and accuracy degradation. Amidst these
challenges, the Lottery Ticket Hypothesis (LTH) emerges as a promising pruning
strategy. It posits that within dense neural networks, there exist winning
tickets or subnetworks that are sparser but do not compromise performance. To
explore a more structure-sparse and energy-saving model, we investigate the
unique synergy of SNNs with LTH and design two novel spiking winning tickets to
push the boundaries of sparsity within SNNs. Furthermore, we introduce an
innovative algorithm capable of simultaneously identifying both weight and
patch-level winning tickets, enabling the achievement of sparser structures
without compromising on the final model's performance. Through comprehensive
experiments on both RGB-based and event-based datasets, we demonstrate that our
spiking lottery ticket achieves comparable or superior performance even when
the model structure is extremely sparse.
Metrics
2 Record Views
Details
- Title
- Pursing the Sparse Limitation of Spiking Deep Learning Structures
- Creators
- Hao ChengJiahang CaoErjia XiaoMengshu SunLe YangJize ZhangXue LinBhavya KailkhuraKaidi XuRenjing Xu
- Publication Details
- arXiv.org
- Resource Type
- Preprint
- Language
- English
- Academic Unit
- Computer Science (Computing)
- Other Identifier
- 991021871366804721