Neural network approximation to nonlinear dynamics by velocity error backpropagation

This paper presents a new type of recurrent neural network (RNN) and its learning algorithm for nonlinear dynamics, called "Velocity-Error Backpropagation (VEBP)." In VEBP, learning is performed in two steps: (a) The velocity vector field of reference trajectories is approximated by a feedforward neural network (NN) with biconnection layers by backpropagating the velocity errors directly. (b) The RNN is constructed by adding integrators and output feedback loops to the trained feedforward NN. VEBP has some advantages over "backpropagation through time (BPTT)," the conventional learning method for RNNs. The effectiveness of the presented RNN and its learning algorithm is demonstrated by simulation results for some examples of nonlinear dynamics.