Power management of wireless sensor nodes with coordinated distributed reinforcement learning

Shaswot Shresthamali, Masaaki Kondo, Hiroshi Nakamura

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Energy Harvesting Wireless Sensor Nodes (EHWSNs) require adaptive energy management policies for uninterrupted perpetual operation in their physical environments. Contemporary online Reinforcement Learning (RL) solutions take an unrealistically long time exploring the environment to converge on working policies. Our work accelerates learning by partitioning the state-space for simultaneous exploration by multiple agents. We achieve this by using a novel coordinated e-greedy method and implement it via Distributed RL (DiRL) in an EHWSN network. Our simulation results show a four-fold increase in state-space penetration and reduction in time to achieve optimal operation by an order of magnitude (50x). Moreover, we also propose methods to reduce instances of disastrous outcomes associated with learning and exploration. This translates to reducing the downtimes of the nodes in simulations corresponding to a real-world scenario by one thirds.

Original languageEnglish
Title of host publicationProceedings - 2019 IEEE International Conference on Computer Design, ICCD 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages638-647
Number of pages10
ISBN (Electronic)9781538666487
DOIs
Publication statusPublished - 2019 Nov
Externally publishedYes
Event37th IEEE International Conference on Computer Design, ICCD 2019 - Abu Dhabi, United Arab Emirates
Duration: 2019 Nov 172019 Nov 20

Publication series

NameProceedings - 2019 IEEE International Conference on Computer Design, ICCD 2019

Conference

Conference37th IEEE International Conference on Computer Design, ICCD 2019
Country/TerritoryUnited Arab Emirates
CityAbu Dhabi
Period19/11/1719/11/20

Keywords

  • Deep Reinforcement Learning
  • Distributed Reinforcement Learning
  • E-greedy exploration
  • Energy Harvesting Wireless Sensor Nodes
  • Energy Neutral Operation
  • Internet of Things
  • Reinforcement Learning

ASJC Scopus subject areas

  • Information Systems and Management
  • Computer Networks and Communications
  • Control and Optimization
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Power management of wireless sensor nodes with coordinated distributed reinforcement learning'. Together they form a unique fingerprint.

Cite this