Multimodal logical inference system for visual-textual entailment

Riko Suzuki, Hitomi Yanaka, Masashi Yoshikawa, Koji Mineshima, Daisuke Bekki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A large amount of research about multimodal inference across text and vision has been recently developed to obtain visually grounded word and sentence representations. In this paper, we use logic-based representations as unified meaning representations for texts and images and present an unsupervised multimodal logical inference system that can effectively prove entailment relations between them. We show that by combining semantic parsing and theorem proving, the system can handle semantically complex sentences for visual-textual inference.

Original languageEnglish
Title of host publicationACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Student Research Workshop
PublisherAssociation for Computational Linguistics (ACL)
Pages386-392
Number of pages7
ISBN (Electronic)9781950737475
Publication statusPublished - 2019
Event57th Annual Meeting of the Association for Computational Linguistics, ACL 2019 - Student Research Workshop, SRW 2019 - Florence, Italy
Duration: 2019 Jul 282019 Aug 2

Publication series

NameACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Student Research Workshop

Conference

Conference57th Annual Meeting of the Association for Computational Linguistics, ACL 2019 - Student Research Workshop, SRW 2019
CountryItaly
CityFlorence
Period19/7/2819/8/2

ASJC Scopus subject areas

  • Language and Linguistics
  • Computer Science(all)
  • Linguistics and Language

Fingerprint Dive into the research topics of 'Multimodal logical inference system for visual-textual entailment'. Together they form a unique fingerprint.

  • Cite this

    Suzuki, R., Yanaka, H., Yoshikawa, M., Mineshima, K., & Bekki, D. (2019). Multimodal logical inference system for visual-textual entailment. In ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Student Research Workshop (pp. 386-392). (ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Student Research Workshop). Association for Computational Linguistics (ACL).