Abstract
Channel pruning for light-weighting networks is very effective in reducing memory footprint and computational cost. Many channel pruning methods assume that the magnitude of a particular element corresponding to each channel reflects the importance of the channel. Unfortunately, such an assumption does not always hold. To solve this problem, this paper proposes a new method to measure the importance of channels based on gradients of mutual information. The proposed method computes and measures gradients of mutual information during back-propagation by arranging a module capable of estimating mutual information. By using the measured statistics as the importance of the channel, less important channels can be removed. Finally, the fine-tuning enables robust performance restoration of the pruned model. Experimental results show that the proposed method provides better performance with smaller parameter sizes and FLOPs than the conventional schemes.
Original language | English |
---|---|
Title of host publication | 2020 IEEE International Conference on Image Processing, ICIP 2020 - Proceedings |
Publisher | IEEE Computer Society |
Pages | 1751-1755 |
Number of pages | 5 |
ISBN (Electronic) | 9781728163956 |
DOIs | |
State | Published - Oct 2020 |
Event | 2020 IEEE International Conference on Image Processing, ICIP 2020 - Virtual, Abu Dhabi, United Arab Emirates Duration: 25 Sep 2020 → 28 Sep 2020 |
Publication series
Name | Proceedings - International Conference on Image Processing, ICIP |
---|---|
Volume | 2020-October |
ISSN (Print) | 1522-4880 |
Conference
Conference | 2020 IEEE International Conference on Image Processing, ICIP 2020 |
---|---|
Country/Territory | United Arab Emirates |
City | Virtual, Abu Dhabi |
Period | 25/09/20 → 28/09/20 |
Bibliographical note
Publisher Copyright:© 2020 IEEE.
Keywords
- convolutional neural network
- model compression
- mutual information
- pruning