Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learning

Hidenori Itaya, Tsubasa Hirakawa, Takayoshi Yamashita, Hironobu Fujiyoshi, Komei Sugiura

研究成果: Conference contribution

2 被引用数 (Scopus)

抄録

Deep reinforcement learning (DRL) has great potential for acquiring the optimal action in complex environments such as games and robot control. However, it is difficult to analyze the decision-making of the agent, i.e., the reasons it selects the action acquired by learning. In this work, we propose Mask-Attention A3C (Mask A3C), which introduces an attention mechanism into Asynchronous Advantage Actor-Critic (A3C), which is an actor-critic-based DRL method, and can analyze the decision-making of an agent in DRL. A3C consists of a feature extractor that extracts features from an image, a policy branch that outputs the policy, and a value branch that outputs the state value. In this method, we focus on the policy and value branches and introduce an attention mechanism into them. The attention mechanism applies a mask processing to the feature maps of each branch using mask-attention that expresses the judgment reason for the policy and state value with a heat map. We visualized mask-attention maps for games on the Atari 2600 and found we could easily analyze the reasons behind an agent's decision-making in various game tasks. Furthermore, experimental results showed that the agent could achieve a higher performance by introducing the attention mechanism.

本文言語English
ホスト出版物のタイトルIJCNN 2021 - International Joint Conference on Neural Networks, Proceedings
出版社Institute of Electrical and Electronics Engineers Inc.
ISBN(電子版)9780738133669
DOI
出版ステータスPublished - 2021 7月 18
イベント2021 International Joint Conference on Neural Networks, IJCNN 2021 - Virtual, Shenzhen, China
継続期間: 2021 7月 182021 7月 22

出版物シリーズ

名前Proceedings of the International Joint Conference on Neural Networks
2021-July

Conference

Conference2021 International Joint Conference on Neural Networks, IJCNN 2021
国/地域China
CityVirtual, Shenzhen
Period21/7/1821/7/22

ASJC Scopus subject areas

  • ソフトウェア
  • 人工知能

フィンガープリント

「Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learning」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル