SeFAct2: Selective Feature Activation for Energy-Efficient CNNs Using Optimized Thresholds

Farhana Sharmin Snigdha; Susmita Dey Manasi; Jiang Hu; Sachin S. Sapatnekar

doi:10.1109/TCAD.2020.3016281

SeFAct2: Selective Feature Activation for Energy-Efficient CNNs Using Optimized Thresholds

Farhana Sharmin Snigdha, Susmita Dey Manasi, Jiang Hu, Sachin S. Sapatnekar

Electrical and Computer Engineering

Research output: Contribution to journal › Article › peer-review

1 Scopus citations

Abstract

This work presents a framework for dynamic energy reduction in hardware accelerators for convolutional neural networks (CNNs). The key idea is based on the early prediction of the features that may be important, with the deactivation of computations related to unimportant features and static bitwidth reduction. The former is applied in late layers of the CNN, while the latter is more effective in the early layers. The procedure includes a methodology for automated threshold tuning to detect feature activation. For various state-of-the-art neural networks, the results show that energy savings of up to about 30% are achievable, after accounting for all implementation overheads, with a small loss in the accuracy.

Original language	English (US)
Article number	9166549
Pages (from-to)	1423-1436
Number of pages	14
Journal	IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Volume	40
Issue number	7
DOIs	https://doi.org/10.1109/TCAD.2020.3016281
State	Published - Jul 2021

Bibliographical note

Publisher Copyright:
© 1982-2012 IEEE.

Keywords

Convolutional neural network (CNN)
deep learning
energy optimization
low-power design
neural network

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access

10.1109/TCAD.2020.3016281

OpenUrl availability

Full text

Cite this

@article{7120486a03ca409daf0ea352bb2f394d,

title = "SeFAct2: Selective Feature Activation for Energy-Efficient CNNs Using Optimized Thresholds",

abstract = "This work presents a framework for dynamic energy reduction in hardware accelerators for convolutional neural networks (CNNs). The key idea is based on the early prediction of the features that may be important, with the deactivation of computations related to unimportant features and static bitwidth reduction. The former is applied in late layers of the CNN, while the latter is more effective in the early layers. The procedure includes a methodology for automated threshold tuning to detect feature activation. For various state-of-the-art neural networks, the results show that energy savings of up to about 30% are achievable, after accounting for all implementation overheads, with a small loss in the accuracy.",

keywords = "Convolutional neural network (CNN), deep learning, energy optimization, low-power design, neural network",

author = "Snigdha, {Farhana Sharmin} and Manasi, {Susmita Dey} and Jiang Hu and Sapatnekar, {Sachin S.}",

note = "Publisher Copyright: {\textcopyright} 1982-2012 IEEE.",

year = "2021",

month = jul,

doi = "10.1109/TCAD.2020.3016281",

language = "English (US)",

volume = "40",

pages = "1423--1436",

journal = "IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems",

issn = "0278-0070",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "7",

}

TY - JOUR

T1 - SeFAct2

T2 - Selective Feature Activation for Energy-Efficient CNNs Using Optimized Thresholds

AU - Snigdha, Farhana Sharmin

AU - Manasi, Susmita Dey

AU - Hu, Jiang

AU - Sapatnekar, Sachin S.

PY - 2021/7

Y1 - 2021/7

N2 - This work presents a framework for dynamic energy reduction in hardware accelerators for convolutional neural networks (CNNs). The key idea is based on the early prediction of the features that may be important, with the deactivation of computations related to unimportant features and static bitwidth reduction. The former is applied in late layers of the CNN, while the latter is more effective in the early layers. The procedure includes a methodology for automated threshold tuning to detect feature activation. For various state-of-the-art neural networks, the results show that energy savings of up to about 30% are achievable, after accounting for all implementation overheads, with a small loss in the accuracy.

AB - This work presents a framework for dynamic energy reduction in hardware accelerators for convolutional neural networks (CNNs). The key idea is based on the early prediction of the features that may be important, with the deactivation of computations related to unimportant features and static bitwidth reduction. The former is applied in late layers of the CNN, while the latter is more effective in the early layers. The procedure includes a methodology for automated threshold tuning to detect feature activation. For various state-of-the-art neural networks, the results show that energy savings of up to about 30% are achievable, after accounting for all implementation overheads, with a small loss in the accuracy.

KW - Convolutional neural network (CNN)

KW - deep learning

KW - energy optimization

KW - low-power design

KW - neural network

UR - http://www.scopus.com/inward/record.url?scp=85099416815&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85099416815&partnerID=8YFLogxK

U2 - 10.1109/TCAD.2020.3016281

DO - 10.1109/TCAD.2020.3016281

M3 - Article

AN - SCOPUS:85099416815

SN - 0278-0070

VL - 40

SP - 1423

EP - 1436

JO - IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

JF - IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

IS - 7

M1 - 9166549

ER -

SeFAct2: Selective Feature Activation for Energy-Efficient CNNs Using Optimized Thresholds

Abstract

Bibliographical note

Keywords

UN SDGs

Access

OpenUrl availability

Other files and links

Fingerprint

Cite this