DsODENet: Neural ODE and Depthwise Separable Convolution for Domain Adaptation on FPGAs

Hiroki Kawakami, Hirohisa Watanabe, Keisuke Sugiura, Hiroki Matsutani

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

High-performance deep neural network (DNN)-based systems are in high demand in edge environments. Due to its high computational complexity, it is challenging to deploy DNNs on edge devices with strict limitations on computational resources. In this paper, we derive a compact while highly-Accurate DNN model, termed dsODENet, by combining recently-proposed parameter reduction techniques: Neural ODE (Ordinary Differential Equation) and DSC (Depthwise Separable Convolution). Neural ODE exploits a similarity between ResNet and ODE, and shares most of weight parameters among multiple layers, which greatly reduces the memory consumption. We apply dsODENet to a domain adaptation as a practical use case with image classification datasets. We also propose a resource-efficient FPGA-based design for dsODENet, where all the parameters and feature maps except for pre-and post-processing layers can be mapped onto onchip memories. It is implemented on Xilinx ZCU104 board and evaluated in terms of domain adaptation accuracy, training speed, FPGA resource utilization, and speedup rate compared to a software counterpart. The results demonstrate that dsODENet achieves comparable or slightly better domain adaptation accuracy compared to our baseline Neural ODE implementation, while the total parameter size without pre-and post-processing layers is reduced by 54.2% to 79.8%. Our FPGA implementation accelerates the inference speed by 27.9 times.

Original languageEnglish
Title of host publicationProceedings - 30th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2022
EditorsArturo Gonzalez-Escribano, Jose Daniel Garcia, Massimo Torquati, Amund Skavhaug
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages152-156
Number of pages5
ISBN (Electronic)9781665469586
DOIs
Publication statusPublished - 2022
Event30th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2022 - Valladolid, Spain
Duration: 2022 Mar 92022 Mar 11

Publication series

NameProceedings - 30th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2022

Conference

Conference30th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2022
Country/TerritorySpain
CityValladolid
Period22/3/922/3/11

Keywords

  • Distillation
  • Domain Adaptation
  • Edge Device
  • FPGA
  • Neural ODE

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems and Management

Fingerprint

Dive into the research topics of 'DsODENet: Neural ODE and Depthwise Separable Convolution for Domain Adaptation on FPGAs'. Together they form a unique fingerprint.

Cite this