Face Commands - User-Defined Facial Gestures for Smart Glasses

Katsutoshi Masai, Kai Kunze, Daisuke Sakamoto, Yuta Sugiura, Maki Sugimoto

研究成果: Conference contribution

抄録

We propose the use of face-related gestures involving the movement of the face, eyes, and head for augmented reality (AR). This technique allows us to use computer systems via hands-free, discreet interactions. In this paper, we present an elicitation study to explore the proper use of facial gestures for daily tasks in the context of a smart home. We used Amazon Mechanical Turk to conduct this study (N=37). Based on the proposed gestures, we report usage scenarios and complexity, proposed associations between gestures/tasks, a user-defined gesture set, and insights from the participants. We also conducted a technical feasibility study (N=13) with participants using smart eyewear to consider their uses in daily life. The device has 16 optical sensors and an inertial measurement unit (IMU). We can potentially integrate the system into optical see-through displays or other smart glasses. The results demonstrate that the device can detect eight temporal face-related gestures with a mean F1 score of 0.911 using a convolutional neural network (CNN). We also report the results of user-independent training and a one-hour recording of the experimenter testing two of the gestures.

本文言語English
ホスト出版物のタイトルProceedings - 2020 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2020
出版社Institute of Electrical and Electronics Engineers Inc.
ページ374-386
ページ数13
ISBN(電子版)9781728185088
DOI
出版ステータスPublished - 2020 11
イベント19th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2020 - Virtual, Recife/Porto de Galinhas, Brazil
継続期間: 2020 11 92020 11 13

出版物シリーズ

名前Proceedings - 2020 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2020

Conference

Conference19th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2020
CountryBrazil
CityVirtual, Recife/Porto de Galinhas
Period20/11/920/11/13

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Software

フィンガープリント 「Face Commands - User-Defined Facial Gestures for Smart Glasses」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル