Automated inference of cognitive performance by fusing multimodal information acquired by smartphone

Takashi Hamatani, Keiichi Ochiai, Akiya Inagaki, Naoki Yamamoto, Yusuke Fukazawa, Masatoshi Kimoto, Kazuki Kiriu, Kouhei Kaminishi, Jun Ota, Yuri Terasawa, Tsukasa Okimura, Takaki Maeda

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recognizing human cognitive performance is important for preserving working efficiency and preventing human error. This paper presents a method for estimating cognitive performance by leveraging multiple information available in a smartphone. The method employs the Go-NoGo task to measure cognitive performance, and fuses contextual and behavioral features to identify the level of performance. It was confirmed that the proposed method could recognize whether cognitive performance was high or low with an average accuracy of 71%, even when only referring to inertial sensor logs. Combining sensing modalities improved the accuracy up to 74%.

Original languageEnglish
Title of host publicationUbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers
PublisherAssociation for Computing Machinery, Inc
Pages921-928
Number of pages8
ISBN (Electronic)9781450368698
DOIs
Publication statusPublished - 2019 Sep 9
Event2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and 2019 ACM International Symposium on Wearable Computers, UbiComp/ISWC 2019 - London, United Kingdom
Duration: 2019 Sep 92019 Sep 13

Publication series

NameUbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers

Conference

Conference2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and 2019 ACM International Symposium on Wearable Computers, UbiComp/ISWC 2019
CountryUnited Kingdom
CityLondon
Period19/9/919/9/13

Fingerprint

Smartphones
Electric fuses
Sensors

Keywords

  • Cognitive performance
  • Go-NoGo task
  • Machine learning
  • Smartphone log

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Cite this

Hamatani, T., Ochiai, K., Inagaki, A., Yamamoto, N., Fukazawa, Y., Kimoto, M., ... Maeda, T. (2019). Automated inference of cognitive performance by fusing multimodal information acquired by smartphone. In UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers (pp. 921-928). (UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers). Association for Computing Machinery, Inc. https://doi.org/10.1145/3341162.3346275

Automated inference of cognitive performance by fusing multimodal information acquired by smartphone. / Hamatani, Takashi; Ochiai, Keiichi; Inagaki, Akiya; Yamamoto, Naoki; Fukazawa, Yusuke; Kimoto, Masatoshi; Kiriu, Kazuki; Kaminishi, Kouhei; Ota, Jun; Terasawa, Yuri; Okimura, Tsukasa; Maeda, Takaki.

UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. Association for Computing Machinery, Inc, 2019. p. 921-928 (UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Hamatani, T, Ochiai, K, Inagaki, A, Yamamoto, N, Fukazawa, Y, Kimoto, M, Kiriu, K, Kaminishi, K, Ota, J, Terasawa, Y, Okimura, T & Maeda, T 2019, Automated inference of cognitive performance by fusing multimodal information acquired by smartphone. in UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, Association for Computing Machinery, Inc, pp. 921-928, 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and 2019 ACM International Symposium on Wearable Computers, UbiComp/ISWC 2019, London, United Kingdom, 19/9/9. https://doi.org/10.1145/3341162.3346275
Hamatani T, Ochiai K, Inagaki A, Yamamoto N, Fukazawa Y, Kimoto M et al. Automated inference of cognitive performance by fusing multimodal information acquired by smartphone. In UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. Association for Computing Machinery, Inc. 2019. p. 921-928. (UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers). https://doi.org/10.1145/3341162.3346275
Hamatani, Takashi ; Ochiai, Keiichi ; Inagaki, Akiya ; Yamamoto, Naoki ; Fukazawa, Yusuke ; Kimoto, Masatoshi ; Kiriu, Kazuki ; Kaminishi, Kouhei ; Ota, Jun ; Terasawa, Yuri ; Okimura, Tsukasa ; Maeda, Takaki. / Automated inference of cognitive performance by fusing multimodal information acquired by smartphone. UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. Association for Computing Machinery, Inc, 2019. pp. 921-928 (UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers).
@inproceedings{2444720b91884c57a68c223addbdadcf,
title = "Automated inference of cognitive performance by fusing multimodal information acquired by smartphone",
abstract = "Recognizing human cognitive performance is important for preserving working efficiency and preventing human error. This paper presents a method for estimating cognitive performance by leveraging multiple information available in a smartphone. The method employs the Go-NoGo task to measure cognitive performance, and fuses contextual and behavioral features to identify the level of performance. It was confirmed that the proposed method could recognize whether cognitive performance was high or low with an average accuracy of 71{\%}, even when only referring to inertial sensor logs. Combining sensing modalities improved the accuracy up to 74{\%}.",
keywords = "Cognitive performance, Go-NoGo task, Machine learning, Smartphone log",
author = "Takashi Hamatani and Keiichi Ochiai and Akiya Inagaki and Naoki Yamamoto and Yusuke Fukazawa and Masatoshi Kimoto and Kazuki Kiriu and Kouhei Kaminishi and Jun Ota and Yuri Terasawa and Tsukasa Okimura and Takaki Maeda",
year = "2019",
month = "9",
day = "9",
doi = "10.1145/3341162.3346275",
language = "English",
series = "UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers",
publisher = "Association for Computing Machinery, Inc",
pages = "921--928",
booktitle = "UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers",

}

TY - GEN

T1 - Automated inference of cognitive performance by fusing multimodal information acquired by smartphone

AU - Hamatani, Takashi

AU - Ochiai, Keiichi

AU - Inagaki, Akiya

AU - Yamamoto, Naoki

AU - Fukazawa, Yusuke

AU - Kimoto, Masatoshi

AU - Kiriu, Kazuki

AU - Kaminishi, Kouhei

AU - Ota, Jun

AU - Terasawa, Yuri

AU - Okimura, Tsukasa

AU - Maeda, Takaki

PY - 2019/9/9

Y1 - 2019/9/9

N2 - Recognizing human cognitive performance is important for preserving working efficiency and preventing human error. This paper presents a method for estimating cognitive performance by leveraging multiple information available in a smartphone. The method employs the Go-NoGo task to measure cognitive performance, and fuses contextual and behavioral features to identify the level of performance. It was confirmed that the proposed method could recognize whether cognitive performance was high or low with an average accuracy of 71%, even when only referring to inertial sensor logs. Combining sensing modalities improved the accuracy up to 74%.

AB - Recognizing human cognitive performance is important for preserving working efficiency and preventing human error. This paper presents a method for estimating cognitive performance by leveraging multiple information available in a smartphone. The method employs the Go-NoGo task to measure cognitive performance, and fuses contextual and behavioral features to identify the level of performance. It was confirmed that the proposed method could recognize whether cognitive performance was high or low with an average accuracy of 71%, even when only referring to inertial sensor logs. Combining sensing modalities improved the accuracy up to 74%.

KW - Cognitive performance

KW - Go-NoGo task

KW - Machine learning

KW - Smartphone log

UR - http://www.scopus.com/inward/record.url?scp=85072881540&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072881540&partnerID=8YFLogxK

U2 - 10.1145/3341162.3346275

DO - 10.1145/3341162.3346275

M3 - Conference contribution

AN - SCOPUS:85072881540

T3 - UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers

SP - 921

EP - 928

BT - UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers

PB - Association for Computing Machinery, Inc

ER -