FPGA Acceleration of ROS2-Based Reinforcement Learning Agents

Daniel Pinheiro Leal, Midori Sugaya, Hideharu Amano, Takeshi Ohkawa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Reinforcement learning agents have shown very good results in robot control and navigation tasks, allowing robots to learn how to interact with an environment appropriately in a model-free manner. However, real-world robot systems have strict latency, power, and cost constraints, thus requiring special hardware consideration for the demanding computations of neural networks. Furthermore, reinforcement learning networks should be able to interface efficiently with the various other robot components. To address these challenges, we propose a method for applying FPGA hardware accelerators to robotics reinforcement learning agents at the inference stage and seamlessly integrating the FPGA hardware module to the robot system by automatically wrapping it in a Robot Operating System 2 (ROS2) node. The proposed system is evaluated in three OpenAI gym control environments: Cartpole-v1, Acrobot-v1, and Pendulum-v0. In the evaluation, both quantized and non-quantized reinforcement learning neural networks are used, and the proposed FPGA system is observed to provide up to a 3.69x speed up and up to 52.7x better performance per watt when compared to an agent running on a ROS2 node on a modern CPU.

Original languageEnglish
Title of host publicationProceedings - 2020 8th International Symposium on Computing and Networking Workshops, CANDARW 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages106-112
Number of pages7
ISBN (Electronic)9781728199191
DOIs
Publication statusPublished - 2020 Nov
Event8th International Symposium on Computing and Networking Workshops, CANDARW 2020 - Virtual, Naha, Japan
Duration: 2020 Nov 242020 Nov 27

Publication series

NameProceedings - 2020 8th International Symposium on Computing and Networking Workshops, CANDARW 2020

Conference

Conference8th International Symposium on Computing and Networking Workshops, CANDARW 2020
Country/TerritoryJapan
CityVirtual, Naha
Period20/11/2420/11/27

Keywords

  • FPGA
  • Hardware Accelerator
  • ROS
  • ROS2
  • Reinforcement Learning
  • Robotics

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Computational Mathematics
  • Control and Optimization

Fingerprint

Dive into the research topics of 'FPGA Acceleration of ROS2-Based Reinforcement Learning Agents'. Together they form a unique fingerprint.

Cite this