Fine-grained action recognition in assembly work scenes by drawing attention to the hands

Takuya Kobayashi, Yoshimitsu Aoki, Shogo Shimizu, Katsuhiro Kusano, Seiji Okumura

研究成果: Conference contribution

1 被引用数 (Scopus)

抄録

The analysis of assembly working scenes is an important tool for improving work efficiency and detecting worker error. Many factories choose to analyze videos of employees working by human observation. It is more efficient, however, to approach this as an action segmentation task, but this is difficult due to the fine-grained actions. In this paper, we focus on featuring the workers hands in action segmentation to describe the detailed moves in assembly work scenes and utilize the tool or product information that the worker is using. We propose two methods to draw attention to the hand from each image in the video: the first is by cutting out the workers hand image and extracting features, and the second is by using an attention module specialized to extract hand features by the training of a regression network. For both methods we combine different types of features - image features and pose features - which are both important information in fine-grained actions. Due to the lack of action datasets that focus on assembly work scenes, we create a new assembly work dataset for our experiments, and the results of those experiments show that our method is effective for analyzing fine-grained action segmentation tasks.

本文言語English
ホスト出版物のタイトルProceedings - 15th International Conference on Signal Image Technology and Internet Based Systems, SISITS 2019
編集者Kokou Yetongnon, Albert Dipanda, Gabriella Sanniti di Baja, Luigi Gallo, Richard Chbeir
出版社Institute of Electrical and Electronics Engineers Inc.
ページ440-446
ページ数7
ISBN(電子版)9781728156866
DOI
出版ステータスPublished - 2019 11
イベント15th International Conference on Signal Image Technology and Internet Based Systems, SISITS 2019 - Sorrento, Italy
継続期間: 2019 11 262019 11 29

出版物シリーズ

名前Proceedings - 15th International Conference on Signal Image Technology and Internet Based Systems, SISITS 2019

Conference

Conference15th International Conference on Signal Image Technology and Internet Based Systems, SISITS 2019
CountryItaly
CitySorrento
Period19/11/2619/11/29

ASJC Scopus subject areas

  • Computer Science Applications
  • Signal Processing
  • Media Technology
  • Computer Networks and Communications

フィンガープリント 「Fine-grained action recognition in assembly work scenes by drawing attention to the hands」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル