A study of artificial neural network architectures for othello evaluation functions

Ksvin J. Binkley, Ken Seehart, Masafumi Hagiwara

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

In this study, we use temporal difference learning (TDL) to investigate the ability of 20 different artificial neural network (ANN) architectures to learn othello game board evaluation functions. The ANN evaluation functions are applied to create a strong othello player using only 1-ply search. In addition to comparing many of the ANN architectures seen in the literature, we introduce several new architectures that consider the game board symmetry. Both embedding the game board symmetry into the network architecture through weight sharing and the outright removal of symmetry through symmetry removal are explored. Experiments varying the number of inputs per game board square from one to three, the number of hidden nodes, and number of hidden layers are also performed. We found it advantageous to consider game board symmetry in the form of symmetry by weight sharing; and that an input encoding of three inputs per square outperformed the one input per square encoding that is commonly seen in the literature. Furthermore, architectures with only one hidden layer were strongly outperformed by architectures with multiple hidden layers. A standard weighted-square board heuristic evaluation function from the literature was used to evaluate the quality of the trained ANN othello players. One of the ANN architectures introduced in this study, an ANN implementing weight sharing and consisting of three hidden layers, using only a 1-ply search, outperformed a weighted-square test heuristic player using a 6-ply minimax search.

Original languageEnglish
Pages (from-to)461-471
Number of pages11
JournalTransactions of the Japanese Society for Artificial Intelligence
Volume22
Issue number5
DOIs
Publication statusPublished - 2007 Jan 1

    Fingerprint

Keywords

  • Artificial neural network
  • Board games
  • Othello
  • Reinforcement learning
  • Temporal difference learning

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Cite this