A study of artificial neural network architectures for othello evaluation functions

Ksvin J. Binkley, Ken Seehart, Masafumi Hagiwara

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

In this study, we use temporal difference learning (TDL) to investigate the ability of 20 different artificial neural network (ANN) architectures to learn othello game board evaluation functions. The ANN evaluation functions are applied to create a strong othello player using only 1-ply search. In addition to comparing many of the ANN architectures seen in the literature, we introduce several new architectures that consider the game board symmetry. Both embedding the game board symmetry into the network architecture through weight sharing and the outright removal of symmetry through symmetry removal are explored. Experiments varying the number of inputs per game board square from one to three, the number of hidden nodes, and number of hidden layers are also performed. We found it advantageous to consider game board symmetry in the form of symmetry by weight sharing; and that an input encoding of three inputs per square outperformed the one input per square encoding that is commonly seen in the literature. Furthermore, architectures with only one hidden layer were strongly outperformed by architectures with multiple hidden layers. A standard weighted-square board heuristic evaluation function from the literature was used to evaluate the quality of the trained ANN othello players. One of the ANN architectures introduced in this study, an ANN implementing weight sharing and consisting of three hidden layers, using only a 1-ply search, outperformed a weighted-square test heuristic player using a 6-ply minimax search.

Original languageEnglish
Pages (from-to)461-471
Number of pages11
JournalTransactions of the Japanese Society for Artificial Intelligence
Volume22
Issue number5
Publication statusPublished - 2007

Fingerprint

Function evaluation
Network architecture
Neural networks
Experiments

Keywords

  • Artificial neural network
  • Board games
  • Othello
  • Reinforcement learning
  • Temporal difference learning

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

A study of artificial neural network architectures for othello evaluation functions. / Binkley, Ksvin J.; Seehart, Ken; Hagiwara, Masafumi.

In: Transactions of the Japanese Society for Artificial Intelligence, Vol. 22, No. 5, 2007, p. 461-471.

Research output: Contribution to journalArticle

@article{79652f713aed4afd8ea8d1491b588472,
title = "A study of artificial neural network architectures for othello evaluation functions",
abstract = "In this study, we use temporal difference learning (TDL) to investigate the ability of 20 different artificial neural network (ANN) architectures to learn othello game board evaluation functions. The ANN evaluation functions are applied to create a strong othello player using only 1-ply search. In addition to comparing many of the ANN architectures seen in the literature, we introduce several new architectures that consider the game board symmetry. Both embedding the game board symmetry into the network architecture through weight sharing and the outright removal of symmetry through symmetry removal are explored. Experiments varying the number of inputs per game board square from one to three, the number of hidden nodes, and number of hidden layers are also performed. We found it advantageous to consider game board symmetry in the form of symmetry by weight sharing; and that an input encoding of three inputs per square outperformed the one input per square encoding that is commonly seen in the literature. Furthermore, architectures with only one hidden layer were strongly outperformed by architectures with multiple hidden layers. A standard weighted-square board heuristic evaluation function from the literature was used to evaluate the quality of the trained ANN othello players. One of the ANN architectures introduced in this study, an ANN implementing weight sharing and consisting of three hidden layers, using only a 1-ply search, outperformed a weighted-square test heuristic player using a 6-ply minimax search.",
keywords = "Artificial neural network, Board games, Othello, Reinforcement learning, Temporal difference learning",
author = "Binkley, {Ksvin J.} and Ken Seehart and Masafumi Hagiwara",
year = "2007",
language = "English",
volume = "22",
pages = "461--471",
journal = "Transactions of the Japanese Society for Artificial Intelligence",
issn = "1346-0714",
publisher = "Japanese Society for Artificial Intelligence",
number = "5",

}

TY - JOUR

T1 - A study of artificial neural network architectures for othello evaluation functions

AU - Binkley, Ksvin J.

AU - Seehart, Ken

AU - Hagiwara, Masafumi

PY - 2007

Y1 - 2007

N2 - In this study, we use temporal difference learning (TDL) to investigate the ability of 20 different artificial neural network (ANN) architectures to learn othello game board evaluation functions. The ANN evaluation functions are applied to create a strong othello player using only 1-ply search. In addition to comparing many of the ANN architectures seen in the literature, we introduce several new architectures that consider the game board symmetry. Both embedding the game board symmetry into the network architecture through weight sharing and the outright removal of symmetry through symmetry removal are explored. Experiments varying the number of inputs per game board square from one to three, the number of hidden nodes, and number of hidden layers are also performed. We found it advantageous to consider game board symmetry in the form of symmetry by weight sharing; and that an input encoding of three inputs per square outperformed the one input per square encoding that is commonly seen in the literature. Furthermore, architectures with only one hidden layer were strongly outperformed by architectures with multiple hidden layers. A standard weighted-square board heuristic evaluation function from the literature was used to evaluate the quality of the trained ANN othello players. One of the ANN architectures introduced in this study, an ANN implementing weight sharing and consisting of three hidden layers, using only a 1-ply search, outperformed a weighted-square test heuristic player using a 6-ply minimax search.

AB - In this study, we use temporal difference learning (TDL) to investigate the ability of 20 different artificial neural network (ANN) architectures to learn othello game board evaluation functions. The ANN evaluation functions are applied to create a strong othello player using only 1-ply search. In addition to comparing many of the ANN architectures seen in the literature, we introduce several new architectures that consider the game board symmetry. Both embedding the game board symmetry into the network architecture through weight sharing and the outright removal of symmetry through symmetry removal are explored. Experiments varying the number of inputs per game board square from one to three, the number of hidden nodes, and number of hidden layers are also performed. We found it advantageous to consider game board symmetry in the form of symmetry by weight sharing; and that an input encoding of three inputs per square outperformed the one input per square encoding that is commonly seen in the literature. Furthermore, architectures with only one hidden layer were strongly outperformed by architectures with multiple hidden layers. A standard weighted-square board heuristic evaluation function from the literature was used to evaluate the quality of the trained ANN othello players. One of the ANN architectures introduced in this study, an ANN implementing weight sharing and consisting of three hidden layers, using only a 1-ply search, outperformed a weighted-square test heuristic player using a 6-ply minimax search.

KW - Artificial neural network

KW - Board games

KW - Othello

KW - Reinforcement learning

KW - Temporal difference learning

UR - http://www.scopus.com/inward/record.url?scp=34250876770&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34250876770&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:34250876770

VL - 22

SP - 461

EP - 471

JO - Transactions of the Japanese Society for Artificial Intelligence

JF - Transactions of the Japanese Society for Artificial Intelligence

SN - 1346-0714

IS - 5

ER -