Neural network approximation to nonlinear dynamics by velocity error backpropagation

Hidenori Ishii, Eitaro Aiyoshi

Research output: Contribution to journalArticle

Abstract

This paper presents a new type of recurrent neural network (RNN) and its learning algorithm for nonlinear dynamics, called "Velocity-Error Backpropagation (VEBP)." In VEBP, learning is performed in two steps: (a) The velocity vector field of reference trajectories is approximated by a feedforward neural network (NN) with biconnection layers by backpropagating the velocity errors directly. (b) The RNN is constructed by adding integrators and output feedback loops to the trained feedforward NN. VEBP has some advantages over "backpropagation through time (BPTT)," the conventional learning method for RNNs. The effectiveness of the presented RNN and its learning algorithm is demonstrated by simulation results for some examples of nonlinear dynamics.

Original languageEnglish
Pages (from-to)26-35
Number of pages10
JournalElectrical Engineering in Japan (English translation of Denki Gakkai Ronbunshi)
Volume140
Issue number2
DOIs
Publication statusPublished - 2002 Jul 30

Fingerprint

Backpropagation
Recurrent neural networks
Neural networks
Feedforward neural networks
Learning algorithms
Trajectories
Feedback

Keywords

  • Approximation
  • Backpropagation
  • Dynamics
  • Learning algorithm
  • Recurrent neural network

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Neural network approximation to nonlinear dynamics by velocity error backpropagation. / Ishii, Hidenori; Aiyoshi, Eitaro.

In: Electrical Engineering in Japan (English translation of Denki Gakkai Ronbunshi), Vol. 140, No. 2, 30.07.2002, p. 26-35.

Research output: Contribution to journalArticle

@article{6387578cffae48c2a71f7798b16a323f,
title = "Neural network approximation to nonlinear dynamics by velocity error backpropagation",
abstract = "This paper presents a new type of recurrent neural network (RNN) and its learning algorithm for nonlinear dynamics, called {"}Velocity-Error Backpropagation (VEBP).{"} In VEBP, learning is performed in two steps: (a) The velocity vector field of reference trajectories is approximated by a feedforward neural network (NN) with biconnection layers by backpropagating the velocity errors directly. (b) The RNN is constructed by adding integrators and output feedback loops to the trained feedforward NN. VEBP has some advantages over {"}backpropagation through time (BPTT),{"} the conventional learning method for RNNs. The effectiveness of the presented RNN and its learning algorithm is demonstrated by simulation results for some examples of nonlinear dynamics.",
keywords = "Approximation, Backpropagation, Dynamics, Learning algorithm, Recurrent neural network",
author = "Hidenori Ishii and Eitaro Aiyoshi",
year = "2002",
month = "7",
day = "30",
doi = "10.1002/eej.10011",
language = "English",
volume = "140",
pages = "26--35",
journal = "Electrical Engineering in Japan (English translation of Denki Gakkai Ronbunshi)",
issn = "0424-7760",
publisher = "John Wiley and Sons Inc.",
number = "2",

}

TY - JOUR

T1 - Neural network approximation to nonlinear dynamics by velocity error backpropagation

AU - Ishii, Hidenori

AU - Aiyoshi, Eitaro

PY - 2002/7/30

Y1 - 2002/7/30

N2 - This paper presents a new type of recurrent neural network (RNN) and its learning algorithm for nonlinear dynamics, called "Velocity-Error Backpropagation (VEBP)." In VEBP, learning is performed in two steps: (a) The velocity vector field of reference trajectories is approximated by a feedforward neural network (NN) with biconnection layers by backpropagating the velocity errors directly. (b) The RNN is constructed by adding integrators and output feedback loops to the trained feedforward NN. VEBP has some advantages over "backpropagation through time (BPTT)," the conventional learning method for RNNs. The effectiveness of the presented RNN and its learning algorithm is demonstrated by simulation results for some examples of nonlinear dynamics.

AB - This paper presents a new type of recurrent neural network (RNN) and its learning algorithm for nonlinear dynamics, called "Velocity-Error Backpropagation (VEBP)." In VEBP, learning is performed in two steps: (a) The velocity vector field of reference trajectories is approximated by a feedforward neural network (NN) with biconnection layers by backpropagating the velocity errors directly. (b) The RNN is constructed by adding integrators and output feedback loops to the trained feedforward NN. VEBP has some advantages over "backpropagation through time (BPTT)," the conventional learning method for RNNs. The effectiveness of the presented RNN and its learning algorithm is demonstrated by simulation results for some examples of nonlinear dynamics.

KW - Approximation

KW - Backpropagation

KW - Dynamics

KW - Learning algorithm

KW - Recurrent neural network

UR - http://www.scopus.com/inward/record.url?scp=0037199358&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0037199358&partnerID=8YFLogxK

U2 - 10.1002/eej.10011

DO - 10.1002/eej.10011

M3 - Article

AN - SCOPUS:0037199358

VL - 140

SP - 26

EP - 35

JO - Electrical Engineering in Japan (English translation of Denki Gakkai Ronbunshi)

JF - Electrical Engineering in Japan (English translation of Denki Gakkai Ronbunshi)

SN - 0424-7760

IS - 2

ER -