Sentiment pen: recognizing emotional context based on handwriting features

Jiawen Han, George Chernyshov, Dingding Zheng, Peizhong Gao, Takuji Narumi, Katrin Wolf, Kai Steven Kunze

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we discuss the assessment of the emotional state of the user from digitized handwriting for implicit human-computer interaction. The proposed concept exemplifies how a digital system could recognize the emotional context of the interaction.We discuss our approach to emotion recognition and the underlying neurophysiological mechanisms. To verify the viability of our approach, we have conducted a series of tests where participants were asked to perform simple writing tasks after being exposed to a series of emotionally-stimulating video clips from EMDB[6], one set of four clips per each quadrant on the circumplex model of emotion[28]. The user-independent Support Vector Classifier (SVC) built using the recorded data shows up to 66% accuracy for certain types of writing tasks for 1 in 4 classification (1. High Valence, High Arousal; 2. High Valence, Low Arousal; 3. Low Valence, High Arousal; 4. Low Valence, Low Arousal). In the same conditions, a user-dependent classifier reaches an average of 70% accuracy across all 12 study participants. While future work is required to improve the classification rate, this work should be seen as proof-of-concept for emotion assessment of users while handwriting aiming to motivate research on implicit interaction while writing to enable emotion-sensitivity in mobile and ubiquitous computing.

Original languageEnglish
Title of host publicationProceedings of the 10th Augmented Human International Conference, AH 2019
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450365475
DOIs
Publication statusPublished - 2019 Mar 11
Externally publishedYes
Event10th Augmented Human International Conference, AH 2019 - Reims, France
Duration: 2019 Mar 112019 Mar 12

Publication series

NameACM International Conference Proceeding Series

Conference

Conference10th Augmented Human International Conference, AH 2019
CountryFrance
CityReims
Period19/3/1119/3/12

Keywords

  • Affective computing
  • Emotional recognition
  • Handwriting analysis

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Fingerprint Dive into the research topics of 'Sentiment pen: recognizing emotional context based on handwriting features'. Together they form a unique fingerprint.

  • Cite this

    Han, J., Chernyshov, G., Zheng, D., Gao, P., Narumi, T., Wolf, K., & Kunze, K. S. (2019). Sentiment pen: recognizing emotional context based on handwriting features. In Proceedings of the 10th Augmented Human International Conference, AH 2019 [a24] (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3311823.3311868