An echo state network with working memories for probabilistic language modeling

Yukinori Homma, Masafumi Hagiwara

研究成果: Conference contribution

1 被引用数 (Scopus)

抄録

In this paper, we propose an ESN having multiple timescale layer and working memories as a probabilistic language model. The reservoir of the proposed model is composed of three neuron groups each with an associated time constant, which enables the model to learn the hierarchical structure of language. We add working memories to enhance the effect of multiple timescale layers. As shown by the experiments, the proposed model can be trained efficiently and accurately to predict the next word from given words. In addition, we found that use of working memories is especially effective in learning grammatical structure.

本文言語English
ホスト出版物のタイトルArtificial Neural Networks and Machine Learning, ICANN 2013 - 23rd International Conference on Artificial Neural Networks, Proceedings
ページ595-602
ページ数8
DOI
出版ステータスPublished - 2013
イベント23rd International Conference on Artificial Neural Networks, ICANN 2013 - Sofia, Bulgaria
継続期間: 2013 9月 102013 9月 13

出版物シリーズ

名前Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
8131 LNCS
ISSN(印刷版)0302-9743
ISSN(電子版)1611-3349

Other

Other23rd International Conference on Artificial Neural Networks, ICANN 2013
国/地域Bulgaria
CitySofia
Period13/9/1013/9/13

ASJC Scopus subject areas

  • 理論的コンピュータサイエンス
  • コンピュータ サイエンス(全般)

フィンガープリント

「An echo state network with working memories for probabilistic language modeling」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル