### Abstract

This paper considers learning by a pulse neural network and proposes a new reinforcement learning algorithm focusing on the ability of pulse neuron elements to process time series. The conventional integrator neuron element is modeled in terms of the average firing rate of the biological neuron. But the pulse neuron is a modeling of the input-output relation of the time-series pulse (spike) and the decay of the internal state (internal potential). The application of such neural networks has been considered in recent engineering studies. It is known in particular that a pulse neuron with a high decay rate acts as a coincidence detector. The proposed model combines pulse neuron elements with different decay rates, which facilitates the processing of the time-series input information and the discrimination of fuzzy states in a partially observable Markov decision process. The proposed network is a four-layered feedforward network in which the pulse neuron elements forming the two hidden layers provide a pseudo-representation of the state in the environment. The elements generate a secondary reinforcement signal which results in learning similar to the conventional reinforcement scheme based on the state evaluation function. A computer experiment verifies that the proposed model works effectively in an environment which is strongly partially observable.

Original language | English |
---|---|

Pages (from-to) | 42-52 |

Number of pages | 11 |

Journal | Systems and Computers in Japan |

Volume | 36 |

Issue number | 3 |

DOIs | |

Publication status | Published - 2005 Mar |

### Fingerprint

### Keywords

- Partially observable Markov decision process
- Pulse neural network
- Reinforcement learning

### ASJC Scopus subject areas

- Hardware and Architecture
- Information Systems
- Theoretical Computer Science
- Computational Theory and Mathematics

### Cite this

**A pulse neural network reinforcement learning algorithm for partially observable Markov decision processes.** / Takita, Koichiro; Hagiwara, Masafumi.

Research output: Contribution to journal › Article

*Systems and Computers in Japan*, vol. 36, no. 3, pp. 42-52. https://doi.org/10.1002/scj.10645

}

TY - JOUR

T1 - A pulse neural network reinforcement learning algorithm for partially observable Markov decision processes

AU - Takita, Koichiro

AU - Hagiwara, Masafumi

PY - 2005/3

Y1 - 2005/3

N2 - This paper considers learning by a pulse neural network and proposes a new reinforcement learning algorithm focusing on the ability of pulse neuron elements to process time series. The conventional integrator neuron element is modeled in terms of the average firing rate of the biological neuron. But the pulse neuron is a modeling of the input-output relation of the time-series pulse (spike) and the decay of the internal state (internal potential). The application of such neural networks has been considered in recent engineering studies. It is known in particular that a pulse neuron with a high decay rate acts as a coincidence detector. The proposed model combines pulse neuron elements with different decay rates, which facilitates the processing of the time-series input information and the discrimination of fuzzy states in a partially observable Markov decision process. The proposed network is a four-layered feedforward network in which the pulse neuron elements forming the two hidden layers provide a pseudo-representation of the state in the environment. The elements generate a secondary reinforcement signal which results in learning similar to the conventional reinforcement scheme based on the state evaluation function. A computer experiment verifies that the proposed model works effectively in an environment which is strongly partially observable.

AB - This paper considers learning by a pulse neural network and proposes a new reinforcement learning algorithm focusing on the ability of pulse neuron elements to process time series. The conventional integrator neuron element is modeled in terms of the average firing rate of the biological neuron. But the pulse neuron is a modeling of the input-output relation of the time-series pulse (spike) and the decay of the internal state (internal potential). The application of such neural networks has been considered in recent engineering studies. It is known in particular that a pulse neuron with a high decay rate acts as a coincidence detector. The proposed model combines pulse neuron elements with different decay rates, which facilitates the processing of the time-series input information and the discrimination of fuzzy states in a partially observable Markov decision process. The proposed network is a four-layered feedforward network in which the pulse neuron elements forming the two hidden layers provide a pseudo-representation of the state in the environment. The elements generate a secondary reinforcement signal which results in learning similar to the conventional reinforcement scheme based on the state evaluation function. A computer experiment verifies that the proposed model works effectively in an environment which is strongly partially observable.

KW - Partially observable Markov decision process

KW - Pulse neural network

KW - Reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=15244342008&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=15244342008&partnerID=8YFLogxK

U2 - 10.1002/scj.10645

DO - 10.1002/scj.10645

M3 - Article

VL - 36

SP - 42

EP - 52

JO - Systems and Computers in Japan

JF - Systems and Computers in Japan

SN - 0882-1666

IS - 3

ER -