Channel Pruning Via Gradient of Mutual Information for Light-Weight Convolutional Neural Networks

Min Kyu Lee, Seunghyun Lee, Sang Hyuk Lee, Byung Cheol Song

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

12 Scopus citations

Abstract

Channel pruning for light-weighting networks is very effective in reducing memory footprint and computational cost. Many channel pruning methods assume that the magnitude of a particular element corresponding to each channel reflects the importance of the channel. Unfortunately, such an assumption does not always hold. To solve this problem, this paper proposes a new method to measure the importance of channels based on gradients of mutual information. The proposed method computes and measures gradients of mutual information during back-propagation by arranging a module capable of estimating mutual information. By using the measured statistics as the importance of the channel, less important channels can be removed. Finally, the fine-tuning enables robust performance restoration of the pruned model. Experimental results show that the proposed method provides better performance with smaller parameter sizes and FLOPs than the conventional schemes.

Original languageEnglish
Title of host publication2020 IEEE International Conference on Image Processing, ICIP 2020 - Proceedings
PublisherIEEE Computer Society
Pages1751-1755
Number of pages5
ISBN (Electronic)9781728163956
DOIs
StatePublished - Oct 2020
Event2020 IEEE International Conference on Image Processing, ICIP 2020 - Virtual, Abu Dhabi, United Arab Emirates
Duration: 25 Sep 202028 Sep 2020

Publication series

NameProceedings - International Conference on Image Processing, ICIP
Volume2020-October
ISSN (Print)1522-4880

Conference

Conference2020 IEEE International Conference on Image Processing, ICIP 2020
Country/TerritoryUnited Arab Emirates
CityVirtual, Abu Dhabi
Period25/09/2028/09/20

Bibliographical note

Publisher Copyright:
© 2020 IEEE.

Keywords

  • convolutional neural network
  • model compression
  • mutual information
  • pruning

Fingerprint

Dive into the research topics of 'Channel Pruning Via Gradient of Mutual Information for Light-Weight Convolutional Neural Networks'. Together they form a unique fingerprint.

Cite this