Facial expression mapping inside head mounted display by embedded optical sensors

Katsuhiro Suzuki; Fumihiko Nakamura; Jiu Otsuka; Katsutoshi Masai; Yuta Itoh; Yuta Sugiura; Maki Sugimoto

doi:10.1145/2984751.2985714

Facial expression mapping inside head mounted display by embedded optical sensors

Katsuhiro Suzuki, Fumihiko Nakamura, Jiu Otsuka, Katsutoshi Masai, Yuta Itoh, Yuta Sugiura, Maki Sugimoto

Department of Information and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

Abstract

Head Mounted Display (HMD) provides an immersive ex-perience in virtual environments for various purposes such as for games and communication. However, it is difficult to capture facial expression in a HMD-based virtual environ-ment because the upper half of user's face is covered up by the HMD. In this paper, we propose a facial expression mapping technology between user and a virtual avatar using embedded optical sensors and machine learning. The dis-tance between each sensor and surface of the face is meas-ured by the optical sensors that are attached inside the HMD. Our system learns the sensor values of each facial expression by neural network and creates a classifier to estimate the current facial expression.

Original language	English
Title of host publication	UIST 2016 Adjunct - Proceedings of the 29th Annual Symposium on User Interface Software and Technology
Publisher	Association for Computing Machinery, Inc
Pages	91-92
Number of pages	2
ISBN (Electronic)	9781450345316
DOIs	https://doi.org/10.1145/2984751.2985714
Publication status	Published - 2016 Oct 16
Event	29th Annual Symposium on User Interface Software and Technology, UIST 2016 - Tokyo, Japan Duration: 2016 Oct 16 → 2016 Oct 19

Other

Other	29th Annual Symposium on User Interface Software and Technology, UIST 2016
Country	Japan
City	Tokyo
Period	16/10/16 → 16/10/19

Keywords

Emotion
Facial Expression Recognition
Neural Network
Photo reflectivity
Virtual Reality
Wearable Sensing

ASJC Scopus subject areas

Software
Human-Computer Interaction

Access to Document

10.1145/2984751.2985714

Fingerprint Dive into the research topics of 'Facial expression mapping inside head mounted display by embedded optical sensors'. Together they form a unique fingerprint.

Cite this

Suzuki, K., Nakamura, F., Otsuka, J., Masai, K., Itoh, Y., Sugiura, Y., & Sugimoto, M. (2016). Facial expression mapping inside head mounted display by embedded optical sensors. In UIST 2016 Adjunct - Proceedings of the 29th Annual Symposium on User Interface Software and Technology (pp. 91-92). Association for Computing Machinery, Inc. https://doi.org/10.1145/2984751.2985714

Facial expression mapping inside head mounted display by embedded optical sensors. / Suzuki, Katsuhiro; Nakamura, Fumihiko; Otsuka, Jiu; Masai, Katsutoshi; Itoh, Yuta; Sugiura, Yuta ; Sugimoto, Maki.

UIST 2016 Adjunct - Proceedings of the 29th Annual Symposium on User Interface Software and Technology. Association for Computing Machinery, Inc, 2016. p. 91-92.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Suzuki, K, Nakamura, F, Otsuka, J, Masai, K, Itoh, Y, Sugiura, Y & Sugimoto, M 2016, Facial expression mapping inside head mounted display by embedded optical sensors. in UIST 2016 Adjunct - Proceedings of the 29th Annual Symposium on User Interface Software and Technology. Association for Computing Machinery, Inc, pp. 91-92, 29th Annual Symposium on User Interface Software and Technology, UIST 2016, Tokyo, Japan, 16/10/16. https://doi.org/10.1145/2984751.2985714

@inproceedings{4afd38ae263944a9aecbfd2a50a12709,

title = "Facial expression mapping inside head mounted display by embedded optical sensors",

abstract = "Head Mounted Display (HMD) provides an immersive ex-perience in virtual environments for various purposes such as for games and communication. However, it is difficult to capture facial expression in a HMD-based virtual environ-ment because the upper half of user's face is covered up by the HMD. In this paper, we propose a facial expression mapping technology between user and a virtual avatar using embedded optical sensors and machine learning. The dis-tance between each sensor and surface of the face is meas-ured by the optical sensors that are attached inside the HMD. Our system learns the sensor values of each facial expression by neural network and creates a classifier to estimate the current facial expression.",

keywords = "Emotion, Facial Expression Recognition, Neural Network, Photo reflectivity, Virtual Reality, Wearable Sensing",

author = "Katsuhiro Suzuki and Fumihiko Nakamura and Jiu Otsuka and Katsutoshi Masai and Yuta Itoh and Yuta Sugiura and Maki Sugimoto",

year = "2016",

month = oct,

day = "16",

doi = "10.1145/2984751.2985714",

language = "English",

pages = "91--92",

booktitle = "UIST 2016 Adjunct - Proceedings of the 29th Annual Symposium on User Interface Software and Technology",

publisher = "Association for Computing Machinery, Inc",

note = "29th Annual Symposium on User Interface Software and Technology, UIST 2016 ; Conference date: 16-10-2016 Through 19-10-2016",

}

TY - GEN

T1 - Facial expression mapping inside head mounted display by embedded optical sensors

AU - Suzuki, Katsuhiro

AU - Nakamura, Fumihiko

AU - Otsuka, Jiu

AU - Masai, Katsutoshi

AU - Itoh, Yuta

AU - Sugiura, Yuta

AU - Sugimoto, Maki

PY - 2016/10/16

Y1 - 2016/10/16

N2 - Head Mounted Display (HMD) provides an immersive ex-perience in virtual environments for various purposes such as for games and communication. However, it is difficult to capture facial expression in a HMD-based virtual environ-ment because the upper half of user's face is covered up by the HMD. In this paper, we propose a facial expression mapping technology between user and a virtual avatar using embedded optical sensors and machine learning. The dis-tance between each sensor and surface of the face is meas-ured by the optical sensors that are attached inside the HMD. Our system learns the sensor values of each facial expression by neural network and creates a classifier to estimate the current facial expression.

AB - Head Mounted Display (HMD) provides an immersive ex-perience in virtual environments for various purposes such as for games and communication. However, it is difficult to capture facial expression in a HMD-based virtual environ-ment because the upper half of user's face is covered up by the HMD. In this paper, we propose a facial expression mapping technology between user and a virtual avatar using embedded optical sensors and machine learning. The dis-tance between each sensor and surface of the face is meas-ured by the optical sensors that are attached inside the HMD. Our system learns the sensor values of each facial expression by neural network and creates a classifier to estimate the current facial expression.

KW - Emotion

KW - Facial Expression Recognition

KW - Neural Network

KW - Photo reflectivity

KW - Virtual Reality

KW - Wearable Sensing

UR - http://www.scopus.com/inward/record.url?scp=84995691175&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84995691175&partnerID=8YFLogxK

U2 - 10.1145/2984751.2985714

DO - 10.1145/2984751.2985714

M3 - Conference contribution

AN - SCOPUS:84995691175

SP - 91

EP - 92

BT - UIST 2016 Adjunct - Proceedings of the 29th Annual Symposium on User Interface Software and Technology

PB - Association for Computing Machinery, Inc

T2 - 29th Annual Symposium on User Interface Software and Technology, UIST 2016

Y2 - 16 October 2016 through 19 October 2016

ER -