Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices

Song Ju Kim, Kaori Ohkoda, Masashi Aono, Hisashi Shima, Makoto Takahashi, Yasuhisa Naitoh, Hiroyuki Akinaga

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Reinforcement learning algorithms are widely used in many practical artificial-intelligence (AI) applications. Herein, we propose a compact hardware-based learning system comprising analogue resistive memory devices called 'resistive analogue neuromorphic devices (RAND).' We begin by showing that the resistance of a RAND linearly varies with the voltage pulses, meaning that a single RAND is sufficient to solve a reinforcement learning problem (i.e. the '2-armed bandit problem') using 'tug-of-war (TOW) dynamics.' Next, we show that 2k-armed bandit problems are also solved by hierarchically combining 2k-1 RANDs. Finally, we numerically demonstrate that the proposed methods are promising as compared with conventional methods.

Original languageEnglish
Title of host publication2019 IEEE International Reliability Physics Symposium, IRPS 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781538695043
DOIs
Publication statusPublished - 2019 May 22
Event2019 IEEE International Reliability Physics Symposium, IRPS 2019 - Monterey, United States
Duration: 2019 Mar 312019 Apr 4

Publication series

NameIEEE International Reliability Physics Symposium Proceedings
Volume2019-March
ISSN (Print)1541-7026

Conference

Conference2019 IEEE International Reliability Physics Symposium, IRPS 2019
CountryUnited States
CityMonterey
Period19/3/3119/4/4

Fingerprint

Reinforcement learning
Learning systems
Learning algorithms
Computer hardware
Artificial intelligence
Data storage equipment
Electric potential

Keywords

  • Decision making
  • Natural computing
  • RAND
  • Reinforcement learning
  • TOW dynamics

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Kim, S. J., Ohkoda, K., Aono, M., Shima, H., Takahashi, M., Naitoh, Y., & Akinaga, H. (2019). Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices. In 2019 IEEE International Reliability Physics Symposium, IRPS 2019 [8720428] (IEEE International Reliability Physics Symposium Proceedings; Vol. 2019-March). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IRPS.2019.8720428

Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices. / Kim, Song Ju; Ohkoda, Kaori; Aono, Masashi; Shima, Hisashi; Takahashi, Makoto; Naitoh, Yasuhisa; Akinaga, Hiroyuki.

2019 IEEE International Reliability Physics Symposium, IRPS 2019. Institute of Electrical and Electronics Engineers Inc., 2019. 8720428 (IEEE International Reliability Physics Symposium Proceedings; Vol. 2019-March).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kim, SJ, Ohkoda, K, Aono, M, Shima, H, Takahashi, M, Naitoh, Y & Akinaga, H 2019, Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices. in 2019 IEEE International Reliability Physics Symposium, IRPS 2019., 8720428, IEEE International Reliability Physics Symposium Proceedings, vol. 2019-March, Institute of Electrical and Electronics Engineers Inc., 2019 IEEE International Reliability Physics Symposium, IRPS 2019, Monterey, United States, 19/3/31. https://doi.org/10.1109/IRPS.2019.8720428
Kim SJ, Ohkoda K, Aono M, Shima H, Takahashi M, Naitoh Y et al. Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices. In 2019 IEEE International Reliability Physics Symposium, IRPS 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8720428. (IEEE International Reliability Physics Symposium Proceedings). https://doi.org/10.1109/IRPS.2019.8720428
Kim, Song Ju ; Ohkoda, Kaori ; Aono, Masashi ; Shima, Hisashi ; Takahashi, Makoto ; Naitoh, Yasuhisa ; Akinaga, Hiroyuki. / Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices. 2019 IEEE International Reliability Physics Symposium, IRPS 2019. Institute of Electrical and Electronics Engineers Inc., 2019. (IEEE International Reliability Physics Symposium Proceedings).
@inproceedings{8fb4d0e9630d489a9dc244cf397fff87,
title = "Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices",
abstract = "Reinforcement learning algorithms are widely used in many practical artificial-intelligence (AI) applications. Herein, we propose a compact hardware-based learning system comprising analogue resistive memory devices called 'resistive analogue neuromorphic devices (RAND).' We begin by showing that the resistance of a RAND linearly varies with the voltage pulses, meaning that a single RAND is sufficient to solve a reinforcement learning problem (i.e. the '2-armed bandit problem') using 'tug-of-war (TOW) dynamics.' Next, we show that 2k-armed bandit problems are also solved by hierarchically combining 2k-1 RANDs. Finally, we numerically demonstrate that the proposed methods are promising as compared with conventional methods.",
keywords = "Decision making, Natural computing, RAND, Reinforcement learning, TOW dynamics",
author = "Kim, {Song Ju} and Kaori Ohkoda and Masashi Aono and Hisashi Shima and Makoto Takahashi and Yasuhisa Naitoh and Hiroyuki Akinaga",
year = "2019",
month = "5",
day = "22",
doi = "10.1109/IRPS.2019.8720428",
language = "English",
series = "IEEE International Reliability Physics Symposium Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2019 IEEE International Reliability Physics Symposium, IRPS 2019",

}

TY - GEN

T1 - Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices

AU - Kim, Song Ju

AU - Ohkoda, Kaori

AU - Aono, Masashi

AU - Shima, Hisashi

AU - Takahashi, Makoto

AU - Naitoh, Yasuhisa

AU - Akinaga, Hiroyuki

PY - 2019/5/22

Y1 - 2019/5/22

N2 - Reinforcement learning algorithms are widely used in many practical artificial-intelligence (AI) applications. Herein, we propose a compact hardware-based learning system comprising analogue resistive memory devices called 'resistive analogue neuromorphic devices (RAND).' We begin by showing that the resistance of a RAND linearly varies with the voltage pulses, meaning that a single RAND is sufficient to solve a reinforcement learning problem (i.e. the '2-armed bandit problem') using 'tug-of-war (TOW) dynamics.' Next, we show that 2k-armed bandit problems are also solved by hierarchically combining 2k-1 RANDs. Finally, we numerically demonstrate that the proposed methods are promising as compared with conventional methods.

AB - Reinforcement learning algorithms are widely used in many practical artificial-intelligence (AI) applications. Herein, we propose a compact hardware-based learning system comprising analogue resistive memory devices called 'resistive analogue neuromorphic devices (RAND).' We begin by showing that the resistance of a RAND linearly varies with the voltage pulses, meaning that a single RAND is sufficient to solve a reinforcement learning problem (i.e. the '2-armed bandit problem') using 'tug-of-war (TOW) dynamics.' Next, we show that 2k-armed bandit problems are also solved by hierarchically combining 2k-1 RANDs. Finally, we numerically demonstrate that the proposed methods are promising as compared with conventional methods.

KW - Decision making

KW - Natural computing

KW - RAND

KW - Reinforcement learning

KW - TOW dynamics

UR - http://www.scopus.com/inward/record.url?scp=85066745105&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066745105&partnerID=8YFLogxK

U2 - 10.1109/IRPS.2019.8720428

DO - 10.1109/IRPS.2019.8720428

M3 - Conference contribution

T3 - IEEE International Reliability Physics Symposium Proceedings

BT - 2019 IEEE International Reliability Physics Symposium, IRPS 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -