Facial expression recognition in daily life by embedded photo reflective sensors on smart eyewear

Research output: Chapter in Book/Report/Conference proceedingConference contribution

22 Citations (Scopus)

Abstract

This paper presents a novel smart eyewear that uses embedded photo reflective sensors and machine learning to recognize a wearer's facial expressions in daily life. We leverage the skin deformation when wearers change their facial expressions. With small photo reflective sensors, we measure the proximity between the skin surface on a face and the eyewear frame where 17 sensors are integrated. A Support Vector Ma- chine (SVM) algorithm was applied for the sensor information. The sensors can cover various facial muscle movements and can be integrated into everyday glasses. The main contributions of our work are as follows. (1) The eyewear recognizes eight facial expressions (92.8% accuracy for one time use and 78.1% for use on 3 different days). (2) It is designed and implemented considering social acceptability. The device looks like normal eyewear, so users can wear it anytime, anywhere. (3) Initial field trials in daily life were undertaken. Our work is one of the first attempts to recognize and evaluate a variety of facial expressions in the form of an unobtrusive wearable device.

Original languageEnglish
Title of host publicationInternational Conference on Intelligent User Interfaces, Proceedings IUI
PublisherAssociation for Computing Machinery
Pages317-326
Number of pages10
Volume07-10-March-2016
ISBN (Print)9781450341370, 9781450341400
DOIs
Publication statusPublished - 2016 Mar 7
Event21st International Conference on Intelligent User Interfaces, IUI 2016 - Sonoma, United States
Duration: 2016 Mar 72016 Mar 10

Other

Other21st International Conference on Intelligent User Interfaces, IUI 2016
CountryUnited States
CitySonoma
Period16/3/716/3/10

Fingerprint

Sensors
Skin
Support vector machines
Muscle
Learning systems
Wear of materials
Glass

Keywords

  • Eyewear Computing
  • Facial Expression Recognition
  • Smart Eye Glasses
  • Wearable

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction

Cite this

Masai, K., Sugiura, Y., Ogata, M., Kunze, K. S., Inami, M., & Sugimoto, M. (2016). Facial expression recognition in daily life by embedded photo reflective sensors on smart eyewear. In International Conference on Intelligent User Interfaces, Proceedings IUI (Vol. 07-10-March-2016, pp. 317-326). Association for Computing Machinery. https://doi.org/10.1145/2856767.2856770

Facial expression recognition in daily life by embedded photo reflective sensors on smart eyewear. / Masai, Katsutoshi; Sugiura, Yuta; Ogata, Masa; Kunze, Kai Steven; Inami, Masahiko; Sugimoto, Maki.

International Conference on Intelligent User Interfaces, Proceedings IUI. Vol. 07-10-March-2016 Association for Computing Machinery, 2016. p. 317-326.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Masai, K, Sugiura, Y, Ogata, M, Kunze, KS, Inami, M & Sugimoto, M 2016, Facial expression recognition in daily life by embedded photo reflective sensors on smart eyewear. in International Conference on Intelligent User Interfaces, Proceedings IUI. vol. 07-10-March-2016, Association for Computing Machinery, pp. 317-326, 21st International Conference on Intelligent User Interfaces, IUI 2016, Sonoma, United States, 16/3/7. https://doi.org/10.1145/2856767.2856770
Masai K, Sugiura Y, Ogata M, Kunze KS, Inami M, Sugimoto M. Facial expression recognition in daily life by embedded photo reflective sensors on smart eyewear. In International Conference on Intelligent User Interfaces, Proceedings IUI. Vol. 07-10-March-2016. Association for Computing Machinery. 2016. p. 317-326 https://doi.org/10.1145/2856767.2856770
Masai, Katsutoshi ; Sugiura, Yuta ; Ogata, Masa ; Kunze, Kai Steven ; Inami, Masahiko ; Sugimoto, Maki. / Facial expression recognition in daily life by embedded photo reflective sensors on smart eyewear. International Conference on Intelligent User Interfaces, Proceedings IUI. Vol. 07-10-March-2016 Association for Computing Machinery, 2016. pp. 317-326
@inproceedings{5180b12b27de45d68dac7409e3a821d4,
title = "Facial expression recognition in daily life by embedded photo reflective sensors on smart eyewear",
abstract = "This paper presents a novel smart eyewear that uses embedded photo reflective sensors and machine learning to recognize a wearer's facial expressions in daily life. We leverage the skin deformation when wearers change their facial expressions. With small photo reflective sensors, we measure the proximity between the skin surface on a face and the eyewear frame where 17 sensors are integrated. A Support Vector Ma- chine (SVM) algorithm was applied for the sensor information. The sensors can cover various facial muscle movements and can be integrated into everyday glasses. The main contributions of our work are as follows. (1) The eyewear recognizes eight facial expressions (92.8{\%} accuracy for one time use and 78.1{\%} for use on 3 different days). (2) It is designed and implemented considering social acceptability. The device looks like normal eyewear, so users can wear it anytime, anywhere. (3) Initial field trials in daily life were undertaken. Our work is one of the first attempts to recognize and evaluate a variety of facial expressions in the form of an unobtrusive wearable device.",
keywords = "Eyewear Computing, Facial Expression Recognition, Smart Eye Glasses, Wearable",
author = "Katsutoshi Masai and Yuta Sugiura and Masa Ogata and Kunze, {Kai Steven} and Masahiko Inami and Maki Sugimoto",
year = "2016",
month = "3",
day = "7",
doi = "10.1145/2856767.2856770",
language = "English",
isbn = "9781450341370",
volume = "07-10-March-2016",
pages = "317--326",
booktitle = "International Conference on Intelligent User Interfaces, Proceedings IUI",
publisher = "Association for Computing Machinery",

}

TY - GEN

T1 - Facial expression recognition in daily life by embedded photo reflective sensors on smart eyewear

AU - Masai, Katsutoshi

AU - Sugiura, Yuta

AU - Ogata, Masa

AU - Kunze, Kai Steven

AU - Inami, Masahiko

AU - Sugimoto, Maki

PY - 2016/3/7

Y1 - 2016/3/7

N2 - This paper presents a novel smart eyewear that uses embedded photo reflective sensors and machine learning to recognize a wearer's facial expressions in daily life. We leverage the skin deformation when wearers change their facial expressions. With small photo reflective sensors, we measure the proximity between the skin surface on a face and the eyewear frame where 17 sensors are integrated. A Support Vector Ma- chine (SVM) algorithm was applied for the sensor information. The sensors can cover various facial muscle movements and can be integrated into everyday glasses. The main contributions of our work are as follows. (1) The eyewear recognizes eight facial expressions (92.8% accuracy for one time use and 78.1% for use on 3 different days). (2) It is designed and implemented considering social acceptability. The device looks like normal eyewear, so users can wear it anytime, anywhere. (3) Initial field trials in daily life were undertaken. Our work is one of the first attempts to recognize and evaluate a variety of facial expressions in the form of an unobtrusive wearable device.

AB - This paper presents a novel smart eyewear that uses embedded photo reflective sensors and machine learning to recognize a wearer's facial expressions in daily life. We leverage the skin deformation when wearers change their facial expressions. With small photo reflective sensors, we measure the proximity between the skin surface on a face and the eyewear frame where 17 sensors are integrated. A Support Vector Ma- chine (SVM) algorithm was applied for the sensor information. The sensors can cover various facial muscle movements and can be integrated into everyday glasses. The main contributions of our work are as follows. (1) The eyewear recognizes eight facial expressions (92.8% accuracy for one time use and 78.1% for use on 3 different days). (2) It is designed and implemented considering social acceptability. The device looks like normal eyewear, so users can wear it anytime, anywhere. (3) Initial field trials in daily life were undertaken. Our work is one of the first attempts to recognize and evaluate a variety of facial expressions in the form of an unobtrusive wearable device.

KW - Eyewear Computing

KW - Facial Expression Recognition

KW - Smart Eye Glasses

KW - Wearable

UR - http://www.scopus.com/inward/record.url?scp=84963757538&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84963757538&partnerID=8YFLogxK

U2 - 10.1145/2856767.2856770

DO - 10.1145/2856767.2856770

M3 - Conference contribution

SN - 9781450341370

SN - 9781450341400

VL - 07-10-March-2016

SP - 317

EP - 326

BT - International Conference on Intelligent User Interfaces, Proceedings IUI

PB - Association for Computing Machinery

ER -