Acquisition of viewpoint transformation and action mappings via sequence to sequence imitative learning by deep neural networks

Ryoichi Nakajo, Shingo Murata, Hiroaki Arie, Tetsuya Ogata

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

We propose an imitative learning model that allows a robot to acquire positional relations between the demonstrator and the robot, and to transform observed actions into robotic actions. Providing robots with imitative capabilities allows us to teach novel actions to them without resorting to trial-and-error approaches. Existing methods for imitative robotic learning require mathematical formulations or conversion modules to translate positional relations between demonstrators and robots. The proposed model uses two neural networks, a convolutional autoencoder (CAE) and a multiple timescale recurrent neural network (MTRNN). The CAE is trained to extract visual features from raw images captured by a camera. The MTRNN is trained to integrate sensory-motor information and to predict next states. We implement this model on a robot and conducted sequence to sequence learning that allows the robot to transform demonstrator actions into robot actions. Through training of the proposedmodel, representations of actions, manipulated objects, and positional relations are formed in the hierarchical structure of the MTRNN. After training, we confirm capability for generating unlearned imitative patterns.

Original languageEnglish
Article number46
JournalFrontiers in Neurorobotics
Volume12
DOIs
Publication statusPublished - 2018 Jan 1
Externally publishedYes

Keywords

  • Deep neural networks
  • Human-robot interaction
  • Imitative learning
  • Recurrent neural networks
  • Sequence to sequence learning

ASJC Scopus subject areas

  • Biomedical Engineering
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Acquisition of viewpoint transformation and action mappings via sequence to sequence imitative learning by deep neural networks'. Together they form a unique fingerprint.

  • Cite this