Boltzmann Machine learning and Mean Field Theory learning with momentum terms

Research output: Contribution to journalArticle

Abstract

This article proposes new algorithms both for Boltzmann Machine (BM) and Mean Field Theory (MFT) learning. They use momentum terms that are derived theoretically to accelerate their learning speeds. The derivation of the new algorithms is based on the following assumptions: (1) The alternate cost function is Gn = Στnζn-τGτ, where Gτ is the information-theoretical measure at the learning time τ, not G which is the commonly used information-theoretical measure in the derivation of BM learning. (2) The most recent weights are assumed in calculating Gn, which technique is used in the derivation of the recursive least-squares algorithm. As a result, momentum terms that accelerate learning can be derived in the BM and MFT learning algorithms. In addition, note that the proposed methods can be used both in batch-mode and pattern-by-pattern learning. Computer simulation is carried out to conform the effectiveness of the proposed MFT algorithm by comparing it with the conventional MFT algorithm.

Original languageEnglish
Pages (from-to)17-25
Number of pages9
JournalJournal of artificial neural networks
Volume2
Issue number1-2
Publication statusPublished - 1995

Fingerprint

Mean field theory
Learning systems
Momentum
Cost functions
Learning algorithms
Computer simulation

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Boltzmann Machine learning and Mean Field Theory learning with momentum terms. / Hagiwara, Masafumi.

In: Journal of artificial neural networks, Vol. 2, No. 1-2, 1995, p. 17-25.

Research output: Contribution to journalArticle

@article{df42363b8be845db90a5bff744daec09,
title = "Boltzmann Machine learning and Mean Field Theory learning with momentum terms",
abstract = "This article proposes new algorithms both for Boltzmann Machine (BM) and Mean Field Theory (MFT) learning. They use momentum terms that are derived theoretically to accelerate their learning speeds. The derivation of the new algorithms is based on the following assumptions: (1) The alternate cost function is Gn = Στnζn-τGτ, where Gτ is the information-theoretical measure at the learning time τ, not G which is the commonly used information-theoretical measure in the derivation of BM learning. (2) The most recent weights are assumed in calculating Gn, which technique is used in the derivation of the recursive least-squares algorithm. As a result, momentum terms that accelerate learning can be derived in the BM and MFT learning algorithms. In addition, note that the proposed methods can be used both in batch-mode and pattern-by-pattern learning. Computer simulation is carried out to conform the effectiveness of the proposed MFT algorithm by comparing it with the conventional MFT algorithm.",
author = "Masafumi Hagiwara",
year = "1995",
language = "English",
volume = "2",
pages = "17--25",
journal = "Journal of artificial neural networks",
issn = "1073-5828",
number = "1-2",

}

TY - JOUR

T1 - Boltzmann Machine learning and Mean Field Theory learning with momentum terms

AU - Hagiwara, Masafumi

PY - 1995

Y1 - 1995

N2 - This article proposes new algorithms both for Boltzmann Machine (BM) and Mean Field Theory (MFT) learning. They use momentum terms that are derived theoretically to accelerate their learning speeds. The derivation of the new algorithms is based on the following assumptions: (1) The alternate cost function is Gn = Στnζn-τGτ, where Gτ is the information-theoretical measure at the learning time τ, not G which is the commonly used information-theoretical measure in the derivation of BM learning. (2) The most recent weights are assumed in calculating Gn, which technique is used in the derivation of the recursive least-squares algorithm. As a result, momentum terms that accelerate learning can be derived in the BM and MFT learning algorithms. In addition, note that the proposed methods can be used both in batch-mode and pattern-by-pattern learning. Computer simulation is carried out to conform the effectiveness of the proposed MFT algorithm by comparing it with the conventional MFT algorithm.

AB - This article proposes new algorithms both for Boltzmann Machine (BM) and Mean Field Theory (MFT) learning. They use momentum terms that are derived theoretically to accelerate their learning speeds. The derivation of the new algorithms is based on the following assumptions: (1) The alternate cost function is Gn = Στnζn-τGτ, where Gτ is the information-theoretical measure at the learning time τ, not G which is the commonly used information-theoretical measure in the derivation of BM learning. (2) The most recent weights are assumed in calculating Gn, which technique is used in the derivation of the recursive least-squares algorithm. As a result, momentum terms that accelerate learning can be derived in the BM and MFT learning algorithms. In addition, note that the proposed methods can be used both in batch-mode and pattern-by-pattern learning. Computer simulation is carried out to conform the effectiveness of the proposed MFT algorithm by comparing it with the conventional MFT algorithm.

UR - http://www.scopus.com/inward/record.url?scp=0029521104&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0029521104&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:0029521104

VL - 2

SP - 17

EP - 25

JO - Journal of artificial neural networks

JF - Journal of artificial neural networks

SN - 1073-5828

IS - 1-2

ER -