Recognition and mapping of facial expressions to avatar by embedded photo reflective sensors in head mounted display

Katsuhiro Suzuki, Fumihiko Nakamura, Jiu Otsuka, Katsutoshi Masai, Yuta Itoh, Yuta Sugiura, Maki Sugimoto

研究成果: Conference contribution

16 被引用数 (Scopus)

抄録

We propose a facial expression mapping technology between virtual avatars and Head-Mounted Display (HMD) users. HMD allow people to enjoy an immersive Virtual Reality (VR) experience. A virtual avatar can be a representative of the user in the virtual environment. However, the synchronization of the the virtual avatar's expressions with those of the HMD user is limited. The major problem of wearing an HMD is that a large portion of the user's face is occluded, making facial recognition difficult in an HMD-based virtual environment. To overcome this problem, we propose a facial expression mapping technology using retro-reflective photoelectric sensors. The sensors attached inside the HMD measures the distance between the sensors and the user's face. The distance values of five basic facial expressions (Neutral, Happy, Angry, Surprised, and Sad) are used for training the neural network to estimate the facial expression of a user. We achieved an overall accuracy of 88% in recognizing the facial expressions. Our system can also reproduce facial expression change in real-time through an existing avatar using regression. Consequently, our system enables estimation and reconstruction of facial expressions that correspond to the user's emotional changes.

本文言語English
ホスト出版物のタイトル2017 IEEE Virtual Reality, VR 2017 - Proceedings
出版社IEEE Computer Society
ページ177-185
ページ数9
ISBN(電子版)9781509066476
DOI
出版ステータスPublished - 2017 4 4
イベント19th IEEE Virtual Reality, VR 2017 - Los Angeles, United States
継続期間: 2017 3 182017 3 22

Other

Other19th IEEE Virtual Reality, VR 2017
CountryUnited States
CityLos Angeles
Period17/3/1817/3/22

ASJC Scopus subject areas

  • Engineering(all)

フィンガープリント 「Recognition and mapping of facial expressions to avatar by embedded photo reflective sensors in head mounted display」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル