Eye-based Interaction Using Embedded Optical Sensors on an Eyewear Device for Facial Expression Recognition

Katsutoshi Masai, Kai Kunze, Maki Sugimoto

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Non-verbal information is essential to understand intentions and emotions and to facilitate social interaction between humans and between humans and computers. One reliable source of such information is the eyes. We investigated the eye-based interaction (recognizing eye gestures or eye movements) using an eyewear device for facial expression recognition. The device incorporates 16 low-cost optical sensors. The system allows hands-free interaction in many situations. Using the device, we evaluated three eye-based interactions. First, we evaluated the accuracy of detecting the gestures with nine participants. The average accuracy of detecting seven different eye gestures is 89.1% with user-dependent training. We used dynamic time warping (DTW) for gesture recognition. Second, we evaluated the accuracy of eye gaze position estimation with five users holding a neutral face. The system showed potential to track the approximate direction of the eyes, with higher accuracy in detecting position y than x. Finally, we did a feasibility study of one user reading jokes while wearing the device. The system was capable of analyzing facial expressions and eye movements in daily contexts.

Original languageEnglish
Title of host publicationProceedings of the Augmented Humans International Conference, AHs 2020
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450376037
DOIs
Publication statusPublished - 2020 Mar 16
Event2020 Augmented Humans International Conference, AHs 2020 - Kaiserslautern, Germany
Duration: 2020 Mar 162020 Mar 17

Publication series

NameACM International Conference Proceeding Series

Conference

Conference2020 Augmented Humans International Conference, AHs 2020
CountryGermany
CityKaiserslautern
Period20/3/1620/3/17

Keywords

  • eyewear computing
  • gaze gesture
  • wearable computing

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Eye-based Interaction Using Embedded Optical Sensors on an Eyewear Device for Facial Expression Recognition'. Together they form a unique fingerprint.

  • Cite this

    Masai, K., Kunze, K., & Sugimoto, M. (2020). Eye-based Interaction Using Embedded Optical Sensors on an Eyewear Device for Facial Expression Recognition. In Proceedings of the Augmented Humans International Conference, AHs 2020 (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3384657.3384787