Visual attribute manipulation using natural language commands

Seito Kasai, Naofumi Akimoto, Masaki Hayashi, Yoshimitsu Aoki

Research output: Contribution to journalArticle

Abstract

In this paper, a novel setting is tackled in which a neural network generates object images with transferred attributes, by conditioning on natural language commands. Conventional methods for object image transformation have used visual attributes, which are components that describe the object's color, posture, etc. This paper builds on this approach and finds an algorithm to precisely extract information from natural language commands, which transfers the attributes of an image and completes this image translation model. The effectiveness of our information extraction model is experimented, with additional tests to see if the change in visual attributes is correctly seen in the image.

Original languageEnglish
Pages (from-to)1102-1109
Number of pages8
JournalSeimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering
Volume85
Issue number12
DOIs
Publication statusPublished - 2019 Jan 1

Keywords

  • Computer vision
  • Convolutional neural network
  • Generative adversarial network
  • Human computer interaction
  • Image editing
  • Image generation
  • Natural language processing
  • Word2vec

ASJC Scopus subject areas

  • Mechanical Engineering

Fingerprint Dive into the research topics of 'Visual attribute manipulation using natural language commands'. Together they form a unique fingerprint.

  • Cite this