Looking Back and Ahead: Adaptation and Planning by Gradient Descent

Shingo Murata, Hiroki Sawa, Shigeki Sugano, Tetsuya Ogata

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Adaptation and planning are crucial for both biological and artificial agents. In this study, we treat these as an inference problem that we solve using a gradient-based optimization approach. We propose adaptation and planning by gradient descent (APGraDe), a gradient-based computational framework with a hierarchical recurrent neural network (RNN) for adaptation and planning. This framework computes (counterfactual) prediction errors by looking back on past situations based on actual observations and by looking ahead to future situations based on preferred observations (or goal). The internal state of the higher level of the RNN is optimized in the direction of minimizing these errors. The errors for the past contribute to the adaptation while errors for the future contribute to the planning. The proposed APGraDe framework is implemented in a humanoid robot and the robot performs a ball manipulation task with a human experimenter. Experimental results show that given a particular preference, the robot can adapt to unexpected situations while pursuing its own preference through the planning of future actions.

Original languageEnglish
Title of host publication2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2019
EditorsAmir Aly, Estela Bicho, Sofiane Boucenna, Bruno Castro da Silva, Mohamed Chetouani, Angel P. del Pobil, Julien Diard, Stephane Doncieux, Tilbe Goksun, Angela Grimminger, Frank Guerin, Yoshinobu Hagiwara, Lorenzo Jamone, Sinan Kalkan, Bruno Lara, Clement Moulin-Frier, Shingo Murata, Takayuki Nagai, Yukie Nagai, Iris Nomikou, Masaki Ogino, Pierre-Yves Oudeyer, Alfredo F. Pereira, Alexandre Pitti, Joanna Raczaszek-Leonardi, Sebastian Risi, Benjamin Rosman, Yulia Sandamirskaya, Malte Schilling, Alessandra Sciutti, Patricia Shaw, Andrea Soltoggio, Michael Spranger, Tadahiro Taniguchi, Serge Thill, Jochen Triesch, Emre Ugur, Anna-Lisa Vollmer
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages151-156
Number of pages6
ISBN (Electronic)9781538681282
DOIs
Publication statusPublished - 2019 Aug
Externally publishedYes
Event9th Joint IEEE International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2019 - Oslo, Norway
Duration: 2019 Aug 192019 Aug 22

Publication series

Name2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2019

Conference

Conference9th Joint IEEE International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2019
CountryNorway
CityOslo
Period19/8/1919/8/22

Keywords

  • active inference
  • free-energy principle
  • planning as inference
  • prediction error minimization
  • predictive coding
  • recurrent neural network

ASJC Scopus subject areas

  • Artificial Intelligence
  • Human-Computer Interaction
  • Control and Optimization

Fingerprint Dive into the research topics of 'Looking Back and Ahead: Adaptation and Planning by Gradient Descent'. Together they form a unique fingerprint.

  • Cite this

    Murata, S., Sawa, H., Sugano, S., & Ogata, T. (2019). Looking Back and Ahead: Adaptation and Planning by Gradient Descent. In A. Aly, E. Bicho, S. Boucenna, B. Castro da Silva, M. Chetouani, A. P. del Pobil, J. Diard, S. Doncieux, T. Goksun, A. Grimminger, F. Guerin, Y. Hagiwara, L. Jamone, S. Kalkan, B. Lara, C. Moulin-Frier, S. Murata, T. Nagai, Y. Nagai, I. Nomikou, M. Ogino, P-Y. Oudeyer, A. F. Pereira, A. Pitti, J. Raczaszek-Leonardi, S. Risi, B. Rosman, Y. Sandamirskaya, M. Schilling, A. Sciutti, P. Shaw, A. Soltoggio, M. Spranger, T. Taniguchi, S. Thill, J. Triesch, E. Ugur, ... A-L. Vollmer (Eds.), 2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2019 (pp. 151-156). [8850693] (2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DEVLRN.2019.8850693