Evaluation of facial expressions as an interaction mechanism and their impact on affect, workload and usability in an AR game

Alan Transon; Adrien Verhulst; Jean Marie Normand; Guillaume Moreau; Maki Sugimoto

doi:10.1109/VSMM.2017.8346263

Evaluation of facial expressions as an interaction mechanism and their impact on affect, workload and usability in an AR game

Alan Transon, Adrien Verhulst, Jean Marie Normand, Guillaume Moreau, Maki Sugimoto

Department of Information and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

6 Citations (Scopus)

Abstract

With the recent development of Head Mounted Display (HMD) for Virtual Reality (VR) allowing to track and recognize user's Facial Expression (FE)s in real-time, we investigated the impact that the use of FEs as an action-trigger input mechanism (e.g. a FE mapped to a single action) has on our emotional state; as well as their workload and usability compared to the use of a controller button. We developed an Augmented Reality (AR)-based memory card where the users select virtual cards using a wand and flip them using either a FE (smiling; frowning) or a Xbox controller button. The users were separated into three groups: (1) flipping the card with a smile (n = 10); (2) flipping the card with a frown (n = 8) and (3) flipping the cards with the Xbox controller button (n = 11). We did not see any significant differences between our groups in: (i) the positive and negative affect of the participants and (ii) the reported workload and usability, thus highlighting that the FEs could be used inside a HMD in the same way as a controller button.

Original language	English
Title of host publication	Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017
Editors	Alonzo Addison, Lizbeth Goodman, Alonzo Addison
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1-8
Number of pages	8
ISBN (Electronic)	9781538644935
DOIs	https://doi.org/10.1109/VSMM.2017.8346263
Publication status	Published - 2018 Apr 24
Event	23rd International Conference on Virtual Systems and Multimedia, VSMM 2017 - Dublin, Ireland Duration: 2017 Oct 31 → 2017 Nov 4

Publication series

Name	Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017
Volume	2018-January

Other

Other	23rd International Conference on Virtual Systems and Multimedia, VSMM 2017
Country/Territory	Ireland
City	Dublin
Period	17/10/31 → 17/11/4

Keywords

Augmented Reality
Facial Expressions
Human computer interaction

ASJC Scopus subject areas

Human-Computer Interaction
Media Technology
Modelling and Simulation

Access to Document

10.1109/VSMM.2017.8346263

Cite this

Transon, A., Verhulst, A., Normand, J. M., Moreau, G., & Sugimoto, M. (2018). Evaluation of facial expressions as an interaction mechanism and their impact on affect, workload and usability in an AR game. In A. Addison, L. Goodman, & A. Addison (Eds.), Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017 (pp. 1-8). (Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017; Vol. 2018-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/VSMM.2017.8346263

Evaluation of facial expressions as an interaction mechanism and their impact on affect, workload and usability in an AR game. / Transon, Alan; Verhulst, Adrien; Normand, Jean Marie et al.

Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017. ed. / Alonzo Addison; Lizbeth Goodman; Alonzo Addison. Institute of Electrical and Electronics Engineers Inc., 2018. p. 1-8 (Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017; Vol. 2018-January).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Transon, A, Verhulst, A, Normand, JM, Moreau, G & Sugimoto, M 2018, Evaluation of facial expressions as an interaction mechanism and their impact on affect, workload and usability in an AR game. in A Addison, L Goodman & A Addison (eds), Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017. Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017, vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 1-8, 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017, Dublin, Ireland, 17/10/31. https://doi.org/10.1109/VSMM.2017.8346263

Transon A, Verhulst A, Normand JM, Moreau G, Sugimoto M. Evaluation of facial expressions as an interaction mechanism and their impact on affect, workload and usability in an AR game. In Addison A, Goodman L, Addison A, editors, Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017. Institute of Electrical and Electronics Engineers Inc. 2018. p. 1-8. (Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017). doi: 10.1109/VSMM.2017.8346263

Transon, Alan ; Verhulst, Adrien ; Normand, Jean Marie et al. / Evaluation of facial expressions as an interaction mechanism and their impact on affect, workload and usability in an AR game. Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017. editor / Alonzo Addison ; Lizbeth Goodman ; Alonzo Addison. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 1-8 (Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017).

@inproceedings{8f679c6668b24b4d954e1f45237d4783,

title = "Evaluation of facial expressions as an interaction mechanism and their impact on affect, workload and usability in an AR game",

abstract = "With the recent development of Head Mounted Display (HMD) for Virtual Reality (VR) allowing to track and recognize user's Facial Expression (FE)s in real-time, we investigated the impact that the use of FEs as an action-trigger input mechanism (e.g. a FE mapped to a single action) has on our emotional state; as well as their workload and usability compared to the use of a controller button. We developed an Augmented Reality (AR)-based memory card where the users select virtual cards using a wand and flip them using either a FE (smiling; frowning) or a Xbox controller button. The users were separated into three groups: (1) flipping the card with a smile (n = 10); (2) flipping the card with a frown (n = 8) and (3) flipping the cards with the Xbox controller button (n = 11). We did not see any significant differences between our groups in: (i) the positive and negative affect of the participants and (ii) the reported workload and usability, thus highlighting that the FEs could be used inside a HMD in the same way as a controller button.",

keywords = "Augmented Reality, Facial Expressions, Human computer interaction",

author = "Alan Transon and Adrien Verhulst and Normand, {Jean Marie} and Guillaume Moreau and Maki Sugimoto",

note = "Funding Information: The second author of this paper was funded during the experiment by an International Research Fellow grant of the Japan Society for the Promotion of Science. Publisher Copyright: {\textcopyright} 2017 IEEE.; 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017 ; Conference date: 31-10-2017 Through 04-11-2017",

year = "2018",

month = apr,

day = "24",

doi = "10.1109/VSMM.2017.8346263",

language = "English",

series = "Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1--8",

editor = "Alonzo Addison and Lizbeth Goodman and Alonzo Addison",

booktitle = "Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017",

}

TY - GEN

T1 - Evaluation of facial expressions as an interaction mechanism and their impact on affect, workload and usability in an AR game

AU - Transon, Alan

AU - Verhulst, Adrien

AU - Normand, Jean Marie

AU - Moreau, Guillaume

AU - Sugimoto, Maki

N1 - Funding Information: The second author of this paper was funded during the experiment by an International Research Fellow grant of the Japan Society for the Promotion of Science. Publisher Copyright: © 2017 IEEE.

PY - 2018/4/24

Y1 - 2018/4/24

N2 - With the recent development of Head Mounted Display (HMD) for Virtual Reality (VR) allowing to track and recognize user's Facial Expression (FE)s in real-time, we investigated the impact that the use of FEs as an action-trigger input mechanism (e.g. a FE mapped to a single action) has on our emotional state; as well as their workload and usability compared to the use of a controller button. We developed an Augmented Reality (AR)-based memory card where the users select virtual cards using a wand and flip them using either a FE (smiling; frowning) or a Xbox controller button. The users were separated into three groups: (1) flipping the card with a smile (n = 10); (2) flipping the card with a frown (n = 8) and (3) flipping the cards with the Xbox controller button (n = 11). We did not see any significant differences between our groups in: (i) the positive and negative affect of the participants and (ii) the reported workload and usability, thus highlighting that the FEs could be used inside a HMD in the same way as a controller button.

AB - With the recent development of Head Mounted Display (HMD) for Virtual Reality (VR) allowing to track and recognize user's Facial Expression (FE)s in real-time, we investigated the impact that the use of FEs as an action-trigger input mechanism (e.g. a FE mapped to a single action) has on our emotional state; as well as their workload and usability compared to the use of a controller button. We developed an Augmented Reality (AR)-based memory card where the users select virtual cards using a wand and flip them using either a FE (smiling; frowning) or a Xbox controller button. The users were separated into three groups: (1) flipping the card with a smile (n = 10); (2) flipping the card with a frown (n = 8) and (3) flipping the cards with the Xbox controller button (n = 11). We did not see any significant differences between our groups in: (i) the positive and negative affect of the participants and (ii) the reported workload and usability, thus highlighting that the FEs could be used inside a HMD in the same way as a controller button.

KW - Augmented Reality

KW - Facial Expressions

KW - Human computer interaction

UR - http://www.scopus.com/inward/record.url?scp=85049743294&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85049743294&partnerID=8YFLogxK

U2 - 10.1109/VSMM.2017.8346263

DO - 10.1109/VSMM.2017.8346263

M3 - Conference contribution

AN - SCOPUS:85049743294

T3 - Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017

SP - 1

EP - 8

BT - Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017

A2 - Addison, Alonzo

A2 - Goodman, Lizbeth

A2 - Addison, Alonzo

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 23rd International Conference on Virtual Systems and Multimedia, VSMM 2017

Y2 - 31 October 2017 through 4 November 2017

ER -

Evaluation of facial expressions as an interaction mechanism and their impact on affect, workload and usability in an AR game

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this