Abstract
In this paper, we propose an acceleration methodology for deep recurrent neural networks (RNNs) implemented on a multi-FPGA platform called Flow-in-Cloud (FiC). RNNs have been proven effective for modeling temporal sequences, such as human speech and written text. However, the implementation of RNNs on traditional hardware is inefficient due to their long-range dependence and irregular computation patterns. This inefficiency manifests itself in the proportional increase of run time with respect to the number of layers of deep RNNs when running on traditional hardware platforms such as a CPUs. Previous works have mostly focused on the optimization of a single RNN cell. In this work, we take advantage of the multi-FPGA system to demonstrate that we can reduce the run time of deep RNNs from O(k) to O(1).
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the 10th International Symposium on Highly-Efficient Accelerators and Reconfigurable Technologies, HEART 2019 |
| Publisher | Association for Computing Machinery |
| ISBN (Electronic) | 9781450372558 |
| DOIs | |
| Publication status | Published - 2019 Jun 6 |
| Event | 10th International Symposium on Highly-Efficient Accelerators and Reconfigurable Technologies, HEART 2019 - Nagasaki, Japan Duration: 2019 Jun 6 → 2019 Jun 7 |
Publication series
| Name | ACM International Conference Proceeding Series |
|---|
Conference
| Conference | 10th International Symposium on Highly-Efficient Accelerators and Reconfigurable Technologies, HEART 2019 |
|---|---|
| Country | Japan |
| City | Nagasaki |
| Period | 19/6/6 → 19/6/7 |
Fingerprint
Keywords
- Acceleration
- FPGAs
- Recurrent Neural Networks
ASJC Scopus subject areas
- Software
- Human-Computer Interaction
- Computer Vision and Pattern Recognition
- Computer Networks and Communications
Cite this
Acceleration of deep recurrent neural networks with an FPGA cluster. / Sun, Yuxi; Ben Ahmed, Akram; Amano, Hideharu.
Proceedings of the 10th International Symposium on Highly-Efficient Accelerators and Reconfigurable Technologies, HEART 2019. Association for Computing Machinery, 2019. 18 (ACM International Conference Proceeding Series).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Acceleration of deep recurrent neural networks with an FPGA cluster
AU - Sun, Yuxi
AU - Ben Ahmed, Akram
AU - Amano, Hideharu
PY - 2019/6/6
Y1 - 2019/6/6
N2 - In this paper, we propose an acceleration methodology for deep recurrent neural networks (RNNs) implemented on a multi-FPGA platform called Flow-in-Cloud (FiC). RNNs have been proven effective for modeling temporal sequences, such as human speech and written text. However, the implementation of RNNs on traditional hardware is inefficient due to their long-range dependence and irregular computation patterns. This inefficiency manifests itself in the proportional increase of run time with respect to the number of layers of deep RNNs when running on traditional hardware platforms such as a CPUs. Previous works have mostly focused on the optimization of a single RNN cell. In this work, we take advantage of the multi-FPGA system to demonstrate that we can reduce the run time of deep RNNs from O(k) to O(1).
AB - In this paper, we propose an acceleration methodology for deep recurrent neural networks (RNNs) implemented on a multi-FPGA platform called Flow-in-Cloud (FiC). RNNs have been proven effective for modeling temporal sequences, such as human speech and written text. However, the implementation of RNNs on traditional hardware is inefficient due to their long-range dependence and irregular computation patterns. This inefficiency manifests itself in the proportional increase of run time with respect to the number of layers of deep RNNs when running on traditional hardware platforms such as a CPUs. Previous works have mostly focused on the optimization of a single RNN cell. In this work, we take advantage of the multi-FPGA system to demonstrate that we can reduce the run time of deep RNNs from O(k) to O(1).
KW - Acceleration
KW - FPGAs
KW - Recurrent Neural Networks
UR - http://www.scopus.com/inward/record.url?scp=85070566081&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85070566081&partnerID=8YFLogxK
U2 - 10.1145/3337801.3337804
DO - 10.1145/3337801.3337804
M3 - Conference contribution
AN - SCOPUS:85070566081
T3 - ACM International Conference Proceeding Series
BT - Proceedings of the 10th International Symposium on Highly-Efficient Accelerators and Reconfigurable Technologies, HEART 2019
PB - Association for Computing Machinery
ER -