Compressing Deep Networks Using Fisher Score of Feature Maps

Mohammadreza Soltani; Suya Wu; Yuerong Li; Robert Ravier; Jie Ding; Vahid Tarokh

doi:10.1109/DCC50243.2021.00083

Compressing Deep Networks Using Fisher Score of Feature Maps

Mohammadreza Soltani, Suya Wu, Yuerong Li, Robert Ravier, Jie Ding, Vahid Tarokh

Statistics (Twin Cities)

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this paper, we propose a new structural technique for pruning deep neural networks with skip-connections. Our approach is based on measuring the importance of feature maps in predicting the output of the model using their Fisher scores. These scores subsequently used for removing the less informative layers from the graph of the network. Extensive experiments on the classification of CIFAR-10, CIFAR-100, and SVHN data sets demonstrate the efficacy of our compressing method both in the number of parameters and operations.

Original language	English (US)
Title of host publication	Proceedings - DCC 2021
Subtitle of host publication	2021 Data Compression Conference
Editors	Ali Bilgin, Michael W. Marcellin, Joan Serra-Sagrista, James A. Storer
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	371
Number of pages	1
ISBN (Electronic)	9780738112275
DOIs	https://doi.org/10.1109/DCC50243.2021.00083
State	Published - Mar 2021
Event	2021 Data Compression Conference, DCC 2021 - Snowbird, United States Duration: Mar 23 2021 → Mar 26 2021

Publication series

Name	Data Compression Conference Proceedings
Volume	2021-March
ISSN (Print)	1068-0314

Conference

Conference	2021 Data Compression Conference, DCC 2021
Country/Territory	United States
City	Snowbird
Period	3/23/21 → 3/26/21

Bibliographical note

Publisher Copyright:
© 2021 IEEE.

Access

10.1109/DCC50243.2021.00083

OpenUrl availability

Full text

Cite this

Soltani, M., Wu, S., Li, Y., Ravier, R., Ding, J., & Tarokh, V. (2021). Compressing Deep Networks Using Fisher Score of Feature Maps. In A. Bilgin, M. W. Marcellin, J. Serra-Sagrista, & J. A. Storer (Eds.), Proceedings - DCC 2021: 2021 Data Compression Conference (pp. 371). Article 9418734 (Data Compression Conference Proceedings; Vol. 2021-March). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DCC50243.2021.00083

Compressing Deep Networks Using Fisher Score of Feature Maps. / Soltani, Mohammadreza; Wu, Suya; Li, Yuerong et al.
Proceedings - DCC 2021: 2021 Data Compression Conference. ed. / Ali Bilgin; Michael W. Marcellin; Joan Serra-Sagrista; James A. Storer. Institute of Electrical and Electronics Engineers Inc., 2021. p. 371 9418734 (Data Compression Conference Proceedings; Vol. 2021-March).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Soltani, M, Wu, S, Li, Y, Ravier, R, Ding, J & Tarokh, V 2021, Compressing Deep Networks Using Fisher Score of Feature Maps. in A Bilgin, MW Marcellin, J Serra-Sagrista & JA Storer (eds), Proceedings - DCC 2021: 2021 Data Compression Conference., 9418734, Data Compression Conference Proceedings, vol. 2021-March, Institute of Electrical and Electronics Engineers Inc., pp. 371, 2021 Data Compression Conference, DCC 2021, Snowbird, United States, 3/23/21. https://doi.org/10.1109/DCC50243.2021.00083

Soltani M, Wu S, Li Y, Ravier R, Ding J, Tarokh V. Compressing Deep Networks Using Fisher Score of Feature Maps. In Bilgin A, Marcellin MW, Serra-Sagrista J, Storer JA, editors, Proceedings - DCC 2021: 2021 Data Compression Conference. Institute of Electrical and Electronics Engineers Inc. 2021. p. 371. 9418734. (Data Compression Conference Proceedings). doi: 10.1109/DCC50243.2021.00083

Soltani, Mohammadreza ; Wu, Suya ; Li, Yuerong et al. / Compressing Deep Networks Using Fisher Score of Feature Maps. Proceedings - DCC 2021: 2021 Data Compression Conference. editor / Ali Bilgin ; Michael W. Marcellin ; Joan Serra-Sagrista ; James A. Storer. Institute of Electrical and Electronics Engineers Inc., 2021. pp. 371 (Data Compression Conference Proceedings).

@inproceedings{a9fce5e5c28545a99c43c9c256da785a,

title = "Compressing Deep Networks Using Fisher Score of Feature Maps",

abstract = "In this paper, we propose a new structural technique for pruning deep neural networks with skip-connections. Our approach is based on measuring the importance of feature maps in predicting the output of the model using their Fisher scores. These scores subsequently used for removing the less informative layers from the graph of the network. Extensive experiments on the classification of CIFAR-10, CIFAR-100, and SVHN data sets demonstrate the efficacy of our compressing method both in the number of parameters and operations.",

author = "Mohammadreza Soltani and Suya Wu and Yuerong Li and Robert Ravier and Jie Ding and Vahid Tarokh",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 2021 Data Compression Conference, DCC 2021 ; Conference date: 23-03-2021 Through 26-03-2021",

year = "2021",

month = mar,

doi = "10.1109/DCC50243.2021.00083",

language = "English (US)",

series = "Data Compression Conference Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "371",

editor = "Ali Bilgin and Marcellin, {Michael W.} and Joan Serra-Sagrista and Storer, {James A.}",

booktitle = "Proceedings - DCC 2021",

}

TY - GEN

T1 - Compressing Deep Networks Using Fisher Score of Feature Maps

AU - Soltani, Mohammadreza

AU - Wu, Suya

AU - Li, Yuerong

AU - Ravier, Robert

AU - Ding, Jie

AU - Tarokh, Vahid

PY - 2021/3

Y1 - 2021/3

N2 - In this paper, we propose a new structural technique for pruning deep neural networks with skip-connections. Our approach is based on measuring the importance of feature maps in predicting the output of the model using their Fisher scores. These scores subsequently used for removing the less informative layers from the graph of the network. Extensive experiments on the classification of CIFAR-10, CIFAR-100, and SVHN data sets demonstrate the efficacy of our compressing method both in the number of parameters and operations.

AB - In this paper, we propose a new structural technique for pruning deep neural networks with skip-connections. Our approach is based on measuring the importance of feature maps in predicting the output of the model using their Fisher scores. These scores subsequently used for removing the less informative layers from the graph of the network. Extensive experiments on the classification of CIFAR-10, CIFAR-100, and SVHN data sets demonstrate the efficacy of our compressing method both in the number of parameters and operations.

UR - http://www.scopus.com/inward/record.url?scp=85106038568&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85106038568&partnerID=8YFLogxK

U2 - 10.1109/DCC50243.2021.00083

DO - 10.1109/DCC50243.2021.00083

M3 - Conference contribution

AN - SCOPUS:85106038568

T3 - Data Compression Conference Proceedings

SP - 371

BT - Proceedings - DCC 2021

A2 - Bilgin, Ali

A2 - Marcellin, Michael W.

A2 - Serra-Sagrista, Joan

A2 - Storer, James A.

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2021 Data Compression Conference, DCC 2021

Y2 - 23 March 2021 through 26 March 2021

ER -

Compressing Deep Networks Using Fisher Score of Feature Maps

Abstract

Publication series

Conference

Bibliographical note

Access

OpenUrl availability

Other files and links

Fingerprint

Cite this