Situation-Aware Emotion Regulation of Conversational Agents with Kinetic Earables

Shin Katayama, Akhil Mathur, Marc Van Den Broeck, Tadashi Okoshi, Jin Nakazawa, Fahim Kawsar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Conversational agents are increasingly becoming digital partners of our everyday computing experiences offering a variety of purposeful information and utility services. Although rich on competency, these agents are entirely oblivious to their users' situational and emotional context today and incapable of adjusting their interaction style and tone contextually. To this end, we present a mixed-method study that informs the design of a situation-and emotion-aware conversational agent for kinetic earables. We surveyed 280 users, and qualitatively interviewed 12 users to understand their expectation from a conversational agent in adapting the interaction style. Grounded on our findings, we develop a first-of-its-kind emotion regulator for a conversational agent on kinetic earable that dynamically adjusts its conversation style, tone, volume in response to users emotional, environmental, social and activity context gathered through speech prosody, motion signals and ambient sound. We describe these context models, the end-to-end system including a purpose-built kinetic earable and their real-world assessment. The experimental results demonstrate that our regulation mechanism invariably elicits better and affective user experience in comparison to baseline conditions in different real-world settings.

Original languageEnglish
Title of host publication2019 8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728138886
DOIs
Publication statusPublished - 2019 Sep
Event8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019 - Cambridge, United Kingdom
Duration: 2019 Sep 32019 Sep 6

Publication series

Name2019 8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019

Conference

Conference8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019
CountryUnited Kingdom
CityCambridge
Period19/9/319/9/6

    Fingerprint

Keywords

  • Context Awareness
  • Conversational Agent
  • Earables
  • Emotion Regulation

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Human-Computer Interaction
  • Behavioral Neuroscience
  • Social Psychology

Cite this

Katayama, S., Mathur, A., Van Den Broeck, M., Okoshi, T., Nakazawa, J., & Kawsar, F. (2019). Situation-Aware Emotion Regulation of Conversational Agents with Kinetic Earables. In 2019 8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019 [8925449] (2019 8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ACII.2019.8925449