Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy

Yosuke Fukuchi, Masahiko Osawa, Hiroshi Yamakawa, Michita Imai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Agents that acquire their own policies autonomously have the risk of accidents caused by the agents’ unexpected behavior. Therefore, it is necessary to improve the predictability of the agents’ behavior in order to ensure the safety. Instruction-based Behavior Explanation (IBE) is a method for a reinforcement learning agent to announce the agent’s future behavior. However, it was not verified that the IBE was applicable to an agent that changes the policy dynamically. In this paper, we consider agents under training and improve the IBE for the application to agents with changing policy. We conducted an experiment to verify if the behavior explanation model of an immature agent worked even after the agent’s further training. The results indicated the applicability of the improved IBE to agents under training.

Original languageEnglish
Title of host publicationNeural Information Processing - 24th International Conference, ICONIP 2017, Proceedings
PublisherSpringer Verlag
Pages100-108
Number of pages9
Volume10634 LNCS
ISBN (Print)9783319700861
DOIs
Publication statusPublished - 2017 Jan 1
Event24th International Conference on Neural Information Processing, ICONIP 2017 - Guangzhou, China
Duration: 2017 Nov 142017 Nov 18

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10634 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other24th International Conference on Neural Information Processing, ICONIP 2017
CountryChina
CityGuangzhou
Period17/11/1417/11/18

Fingerprint

Reinforcement learning
Reinforcement Learning
Policy
Predictability
Accidents
Safety
Verify

Keywords

  • Instruction-based behavior explanation (IBE)
  • Reinforcement learning

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Fukuchi, Y., Osawa, M., Yamakawa, H., & Imai, M. (2017). Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy. In Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings (Vol. 10634 LNCS, pp. 100-108). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10634 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-70087-8_11

Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy. / Fukuchi, Yosuke; Osawa, Masahiko; Yamakawa, Hiroshi; Imai, Michita.

Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings. Vol. 10634 LNCS Springer Verlag, 2017. p. 100-108 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10634 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Fukuchi, Y, Osawa, M, Yamakawa, H & Imai, M 2017, Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy. in Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings. vol. 10634 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10634 LNCS, Springer Verlag, pp. 100-108, 24th International Conference on Neural Information Processing, ICONIP 2017, Guangzhou, China, 17/11/14. https://doi.org/10.1007/978-3-319-70087-8_11
Fukuchi Y, Osawa M, Yamakawa H, Imai M. Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy. In Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings. Vol. 10634 LNCS. Springer Verlag. 2017. p. 100-108. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-70087-8_11
Fukuchi, Yosuke ; Osawa, Masahiko ; Yamakawa, Hiroshi ; Imai, Michita. / Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy. Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings. Vol. 10634 LNCS Springer Verlag, 2017. pp. 100-108 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{7aba8d8f337e448fa98b63e09feef22f,
title = "Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy",
abstract = "Agents that acquire their own policies autonomously have the risk of accidents caused by the agents’ unexpected behavior. Therefore, it is necessary to improve the predictability of the agents’ behavior in order to ensure the safety. Instruction-based Behavior Explanation (IBE) is a method for a reinforcement learning agent to announce the agent’s future behavior. However, it was not verified that the IBE was applicable to an agent that changes the policy dynamically. In this paper, we consider agents under training and improve the IBE for the application to agents with changing policy. We conducted an experiment to verify if the behavior explanation model of an immature agent worked even after the agent’s further training. The results indicated the applicability of the improved IBE to agents under training.",
keywords = "Instruction-based behavior explanation (IBE), Reinforcement learning",
author = "Yosuke Fukuchi and Masahiko Osawa and Hiroshi Yamakawa and Michita Imai",
year = "2017",
month = "1",
day = "1",
doi = "10.1007/978-3-319-70087-8_11",
language = "English",
isbn = "9783319700861",
volume = "10634 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "100--108",
booktitle = "Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings",
address = "Germany",

}

TY - GEN

T1 - Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy

AU - Fukuchi, Yosuke

AU - Osawa, Masahiko

AU - Yamakawa, Hiroshi

AU - Imai, Michita

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Agents that acquire their own policies autonomously have the risk of accidents caused by the agents’ unexpected behavior. Therefore, it is necessary to improve the predictability of the agents’ behavior in order to ensure the safety. Instruction-based Behavior Explanation (IBE) is a method for a reinforcement learning agent to announce the agent’s future behavior. However, it was not verified that the IBE was applicable to an agent that changes the policy dynamically. In this paper, we consider agents under training and improve the IBE for the application to agents with changing policy. We conducted an experiment to verify if the behavior explanation model of an immature agent worked even after the agent’s further training. The results indicated the applicability of the improved IBE to agents under training.

AB - Agents that acquire their own policies autonomously have the risk of accidents caused by the agents’ unexpected behavior. Therefore, it is necessary to improve the predictability of the agents’ behavior in order to ensure the safety. Instruction-based Behavior Explanation (IBE) is a method for a reinforcement learning agent to announce the agent’s future behavior. However, it was not verified that the IBE was applicable to an agent that changes the policy dynamically. In this paper, we consider agents under training and improve the IBE for the application to agents with changing policy. We conducted an experiment to verify if the behavior explanation model of an immature agent worked even after the agent’s further training. The results indicated the applicability of the improved IBE to agents under training.

KW - Instruction-based behavior explanation (IBE)

KW - Reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85035146881&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85035146881&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-70087-8_11

DO - 10.1007/978-3-319-70087-8_11

M3 - Conference contribution

AN - SCOPUS:85035146881

SN - 9783319700861

VL - 10634 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 100

EP - 108

BT - Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings

PB - Springer Verlag

ER -