Slightly-slacked dropout for improving neural network learning on FPGA

Sota Sawaguchi; Hiroaki Nishi

doi:10.1016/j.icte.2018.04.006

Slightly-slacked dropout for improving neural network learning on FPGA

Sota Sawaguchi, Hiroaki Nishi

Department of System Design Engineering

Research output: Contribution to journal › Article › peer-review

6 Citations (Scopus)

Abstract

Neural Network Learning (NNL) is compute-intensive. It often involves a dropout technique which effectively regularizes the network to avoid overfitting. As such, a hardware accelerator for dropout NNL has been proposed; however, the existing method encounters a huge transfer cost between hardware and software. This paper proposes Slightly-Slacked Dropout (SS-Dropout), a novel deterministic dropout technique to address the transfer cost while accelerating the process. Experimental results show that our SS-Dropout technique improves both the usual and dropout NNL accelerator, i.e., 1.55 times speed-up and three order-of-magnitude less transfer cost, respectively.

Original language	English
Pages (from-to)	75-80
Number of pages	6
Journal	ICT Express
Volume	4
Issue number	2
DOIs	https://doi.org/10.1016/j.icte.2018.04.006
Publication status	Published - 2018 Jun

Keywords

Dropout technique
Mini-batch SGD algorithm
Neural Network
SoC FPGA

ASJC Scopus subject areas

Software
Information Systems
Hardware and Architecture
Computer Networks and Communications
Artificial Intelligence

Access to Document

10.1016/j.icte.2018.04.006

Cite this

@article{9f8be9ab501e43d68b3b3242422d8ad2,

title = "Slightly-slacked dropout for improving neural network learning on FPGA",

abstract = "Neural Network Learning (NNL) is compute-intensive. It often involves a dropout technique which effectively regularizes the network to avoid overfitting. As such, a hardware accelerator for dropout NNL has been proposed; however, the existing method encounters a huge transfer cost between hardware and software. This paper proposes Slightly-Slacked Dropout (SS-Dropout), a novel deterministic dropout technique to address the transfer cost while accelerating the process. Experimental results show that our SS-Dropout technique improves both the usual and dropout NNL accelerator, i.e., 1.55 times speed-up and three order-of-magnitude less transfer cost, respectively.",

keywords = "Dropout technique, Mini-batch SGD algorithm, Neural Network, SoC FPGA",

author = "Sota Sawaguchi and Hiroaki Nishi",

note = "Funding Information: This work was partially supported by MEXT/JSPS KAKENHI Grant (B) Number JP16H04455 and JP17H01739 , through funding received from SECOM Science and Technology Foundation of the R&D project “Design of Information and Communication Platform for Future Smart Community Services” by the Ministry of Internal Affairs and Communications of Japan, and from Keio Univ. Global Smart Society Creation Project Research. Publisher Copyright: {\textcopyright} 2018 The Korean Institute of Communications and Information Sciences (KICS)",

year = "2018",

month = jun,

doi = "10.1016/j.icte.2018.04.006",

language = "English",

volume = "4",

pages = "75--80",

journal = "ICT Express",

issn = "2405-9595",

publisher = "Korean Institute of Communications Information Sciences",

number = "2",

}

TY - JOUR

T1 - Slightly-slacked dropout for improving neural network learning on FPGA

AU - Sawaguchi, Sota

AU - Nishi, Hiroaki

N1 - Funding Information: This work was partially supported by MEXT/JSPS KAKENHI Grant (B) Number JP16H04455 and JP17H01739 , through funding received from SECOM Science and Technology Foundation of the R&D project “Design of Information and Communication Platform for Future Smart Community Services” by the Ministry of Internal Affairs and Communications of Japan, and from Keio Univ. Global Smart Society Creation Project Research. Publisher Copyright: © 2018 The Korean Institute of Communications and Information Sciences (KICS)

PY - 2018/6

Y1 - 2018/6

N2 - Neural Network Learning (NNL) is compute-intensive. It often involves a dropout technique which effectively regularizes the network to avoid overfitting. As such, a hardware accelerator for dropout NNL has been proposed; however, the existing method encounters a huge transfer cost between hardware and software. This paper proposes Slightly-Slacked Dropout (SS-Dropout), a novel deterministic dropout technique to address the transfer cost while accelerating the process. Experimental results show that our SS-Dropout technique improves both the usual and dropout NNL accelerator, i.e., 1.55 times speed-up and three order-of-magnitude less transfer cost, respectively.

AB - Neural Network Learning (NNL) is compute-intensive. It often involves a dropout technique which effectively regularizes the network to avoid overfitting. As such, a hardware accelerator for dropout NNL has been proposed; however, the existing method encounters a huge transfer cost between hardware and software. This paper proposes Slightly-Slacked Dropout (SS-Dropout), a novel deterministic dropout technique to address the transfer cost while accelerating the process. Experimental results show that our SS-Dropout technique improves both the usual and dropout NNL accelerator, i.e., 1.55 times speed-up and three order-of-magnitude less transfer cost, respectively.

KW - Dropout technique

KW - Mini-batch SGD algorithm

KW - Neural Network

KW - SoC FPGA

UR - http://www.scopus.com/inward/record.url?scp=85046683407&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046683407&partnerID=8YFLogxK

U2 - 10.1016/j.icte.2018.04.006

DO - 10.1016/j.icte.2018.04.006

M3 - Article

AN - SCOPUS:85046683407

VL - 4

SP - 75

EP - 80

JO - ICT Express

JF - ICT Express

SN - 2405-9595

IS - 2

ER -

Slightly-slacked dropout for improving neural network learning on FPGA

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this