Accelerating ODE-Based Neural Networks on Low-Cost FPGAs

Hirohisa Watanabe, Hiroki Matsutani

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

ODENet is a deep neural network architecture in which a stacking structure of ResNet is implemented with an ordinary differential equation (ODE) solver. It can reduce the number of parameters and strike a balance between accuracy and performance by selecting a proper solver. It is also possible to improve the accuracy while keeping the same number of parameters on resource-limited edge devices. In this paper, using Euler method as an ODE solver, a part of ODENet is implemented as a dedicated logic on a low-cost FPGA (Field-Programmable Gate Array) board, such as PYNQ-Z2 board. As ODENet variants, reduced ODENets (rODENets) each of which heavily uses a part of ODENet layers and reduces/eliminates some layers differently are proposed and analyzed for low-cost FPGA implementation. They are evaluated in terms of parameter size, accuracy, execution time, and resource utilization on the FPGA. The results show that an overall execution time of an rODENet variant is improved by up to 2.66 times compared to a pure software execution while keeping a comparable accuracy to the original ODENet.

Original languageEnglish
Title of host publication2021 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2021 - In conjunction with IEEE IPDPS 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages88-95
Number of pages8
ISBN (Electronic)9781665435772
DOIs
Publication statusPublished - 2021 Jun
Event2021 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2021 - Virtual, Portland, United States
Duration: 2021 May 17 → …

Publication series

Name2021 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2021 - In conjunction with IEEE IPDPS 2021

Conference

Conference2021 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2021
Country/TerritoryUnited States
CityVirtual, Portland
Period21/5/17 → …

Keywords

  • CNN
  • FPGA
  • Neural network
  • Neural ODE
  • ODE

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems

Fingerprint

Dive into the research topics of 'Accelerating ODE-Based Neural Networks on Low-Cost FPGAs'. Together they form a unique fingerprint.

Cite this