SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics

Hitomi Yanaka, Koji Mineshima, Kentaro Inui

研究成果: Conference contribution

1 被引用数 (Scopus)

抄録

Recently, deep neural networks (DNNs) have achieved great success in semantically challenging NLP tasks, yet it remains unclear whether DNN models can capture compositional meanings, those aspects of meaning that have been long studied in formal semantics. To investigate this issue, we propose a Systematic Generalization testbed based on Natural language Semantics (SyGNS), whose challenge is to map natural language sentences to multiple forms of scoped meaning representations, designed to account for various semantic phenomena. Using SyGNS, we test whether neural networks can systematically parse sentences involving novel combinations of logical expressions such as quantifiers and negation. Experiments show that Transformer and GRU models can generalize to unseen combinations of quantifiers, negations, and modifiers that are similar to given training instances in form, but not to the others. We also find that the generalization performance to unseen combinations is better when the form of meaning representations is simpler. The data and code for SyGNS are publicly available at https://github.com/verypluming/SyGNS.

本文言語English
ホスト出版物のタイトルFindings of the Association for Computational Linguistics
ホスト出版物のサブタイトルACL-IJCNLP 2021
編集者Chengqing Zong, Fei Xia, Wenjie Li, Roberto Navigli
出版社Association for Computational Linguistics (ACL)
ページ103-119
ページ数17
ISBN(電子版)9781954085541
出版ステータスPublished - 2021
イベントFindings of the Association for Computational Linguistics: ACL-IJCNLP 2021 - Virtual, Online
継続期間: 2021 8月 12021 8月 6

出版物シリーズ

名前Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

Conference

ConferenceFindings of the Association for Computational Linguistics: ACL-IJCNLP 2021
CityVirtual, Online
Period21/8/121/8/6

ASJC Scopus subject areas

  • 言語および言語学
  • 言語学および言語

フィンガープリント

「SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル