Facial expression mapping inside head mounted display by embedded optical sensors

Katsuhiro Suzuki, Fumihiko Nakamura, Jiu Otsuka, Katsutoshi Masai, Yuta Itoh, Yuta Sugiura, Maki Sugimoto

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Head Mounted Display (HMD) provides an immersive ex-perience in virtual environments for various purposes such as for games and communication. However, it is difficult to capture facial expression in a HMD-based virtual environ-ment because the upper half of user's face is covered up by the HMD. In this paper, we propose a facial expression mapping technology between user and a virtual avatar using embedded optical sensors and machine learning. The dis-tance between each sensor and surface of the face is meas-ured by the optical sensors that are attached inside the HMD. Our system learns the sensor values of each facial expression by neural network and creates a classifier to estimate the current facial expression.

Original languageEnglish
Title of host publicationUIST 2016 Adjunct - Proceedings of the 29th Annual Symposium on User Interface Software and Technology
PublisherAssociation for Computing Machinery, Inc
Pages91-92
Number of pages2
ISBN (Electronic)9781450345316
DOIs
Publication statusPublished - 2016 Oct 16
Event29th Annual Symposium on User Interface Software and Technology, UIST 2016 - Tokyo, Japan
Duration: 2016 Oct 162016 Oct 19

Other

Other29th Annual Symposium on User Interface Software and Technology, UIST 2016
CountryJapan
CityTokyo
Period16/10/1616/10/19

    Fingerprint

Keywords

  • Emotion
  • Facial Expression Recognition
  • Neural Network
  • Photo reflectivity
  • Virtual Reality
  • Wearable Sensing

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction

Cite this

Suzuki, K., Nakamura, F., Otsuka, J., Masai, K., Itoh, Y., Sugiura, Y., & Sugimoto, M. (2016). Facial expression mapping inside head mounted display by embedded optical sensors. In UIST 2016 Adjunct - Proceedings of the 29th Annual Symposium on User Interface Software and Technology (pp. 91-92). Association for Computing Machinery, Inc. https://doi.org/10.1145/2984751.2985714