Analysis of momentum term in back-propagation

Masafumi Hagiwara, Akira Sato

研究成果: Article査読

9 被引用数 (Scopus)

抄録

The back-propagation algorithm has been applied to many fields, and has shown large capability of neural networks. Many people use the back-propagation algorithm together with a momentum term to accelerate its convergence. However, in spite of the importance for theoretical studies, theoretical background of a momentum term has been unknown so far. First, this paper explains clearly the theoretical origin of a momentum term in the back-propagation algorithm for both a batch mode learning and a pattern-by-pattern learning. We will prove that the back-propagation algorithm having a momentum term can be derived through the following two assumptions: 1) The cost function is En = n/Σ/μ αn-μ Eμ, where Eμ is the summation of squared error at the output layer at the μth learning time and a is the momentum coefficient. 2) The latest weights are assumed in calculating the cost function En. Next, we derive a simple relationship between momentum, learning rate, and learning speed and then further discussion is made with computer simulation.

本文言語English
ページ(範囲)1080-1086
ページ数7
ジャーナルIEICE Transactions on Information and Systems
E78-D
8
出版ステータスPublished - 1995 8月 1

ASJC Scopus subject areas

  • ソフトウェア
  • ハードウェアとアーキテクチャ
  • コンピュータ ビジョンおよびパターン認識
  • 電子工学および電気工学
  • 人工知能

フィンガープリント

「Analysis of momentum term in back-propagation」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル