Overview of AIWolfDial 2019 Shared Task: Contest of automatic dialog agents to play the werewolf game through conversations

Yoshinobu Kano, Claus Aranha, Michimasa Inaba, Fujio Toriumi, Hirotaka Osawa, Daisuke Katagami, Takashi Otsuki, Issei Tsunoda, Shoji Nagayama, Dolça Tellols, Yu Sugawara, Yohei Nakata

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The AIWolf project has been holding contests to play the Werewolf game (“Mafia”) by automatic agents for a couple of years. A difficulty of the Werewolf game is that the game is an imperfect information game, where a player’s role is hidden from other players. Players are required to infer the roles of other players through free conversations; players of a specific role should tell a lie, while others try to break through lies. We employ this werewolf game as a novel way of evaluations for dialog systems. Because the werewolf game forces players to deceive, persuade, and detect lies, neither inconsistent nor vague response are evaluated as “unnatural”, losing in the game. Our werewolf game competition and evaluation could be a new interesting evaluation criteria for dialog systems, but also for imperfect information game theories. In addition, the werewolf game allows any conversation, so the game includes both task-oriented and non-task-oriented conversations. This aspect would provide a handy intermediate goal rather than to create a general dialog system from scratch. In this AIWolfDial 2019 shared task, five participant agents played games in English and Japanese. We performed subjective evaluations on these game logs.

Original languageEnglish
Title of host publicationAIWolfDial 2019 - Proceedings of the 1st International Workshop of AI Werewolf and Dialog System, Proceedings of the Workshop
PublisherAssociation for Computational Linguistics (ACL)
Pages1-5
Number of pages5
ISBN (Electronic)9781948087919
Publication statusPublished - 2019
Externally publishedYes
Event1st International Workshop of AI Werewolf and Dialog System, AIWolfDial 2019, in conjunction with INLG 2019 - Tokyo, Japan
Duration: 2019 Oct 29 → …

Publication series

NameAIWolfDial 2019 - Proceedings of the 1st International Workshop of AI Werewolf and Dialog System, Proceedings of the Workshop

Conference

Conference1st International Workshop of AI Werewolf and Dialog System, AIWolfDial 2019, in conjunction with INLG 2019
Country/TerritoryJapan
CityTokyo
Period19/10/29 → …

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Overview of AIWolfDial 2019 Shared Task: Contest of automatic dialog agents to play the werewolf game through conversations'. Together they form a unique fingerprint.

Cite this