A multimodal classifier generative adversarial network for carry and place tasks from ambiguous language instructions

Aly Magassouba, Komei Sugiura, Hisashi Kawai

研究成果: Article査読

12 被引用数 (Scopus)

抄録

This letter focuses on a multimodal language understanding method for carry-and-place tasks with domestic service robots. We address the case of ambiguous instructions, that is, when the target area is not specified. For instance 'put away the milk and cereal' is a natural instruction where there is ambiguity regarding the target area, considering environments in daily life. Conventionally, this instruction can be disambiguated from a dialogue system, but at the cost of time and cumbersome interaction. Instead, we propose a multimodal approach, in which the instructions are disambiguated using the robot's state and environment context. We develop the Multi-Modal Classifier Generative Adversarial Network (MMC-GAN) to predict the likelihood of different target areas considering the robot's physical limitation and the target clutter. Our approach, MMC-GAN, significantly improves accuracy compared with baseline methods that use instructions only or simple deep neural networks.

本文言語English
ページ(範囲)3113-3120
ページ数8
ジャーナルIEEE Robotics and Automation Letters
3
4
DOI
出版ステータスPublished - 2018 10
外部発表はい

ASJC Scopus subject areas

  • 制御およびシステム工学
  • 生体医工学
  • 人間とコンピュータの相互作用
  • 機械工学
  • コンピュータ ビジョンおよびパターン認識
  • コンピュータ サイエンスの応用
  • 制御と最適化
  • 人工知能

フィンガープリント

「A multimodal classifier generative adversarial network for carry and place tasks from ambiguous language instructions」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル