An augmented e-reader for multimodal literacy

Heng Gu, Susana Sanchez, Kai Steven Kunze, Masahiko Inami

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

We present an initial prototype to extend a reading tablet for haptic feedback -with a surface vibration transducer. This, in order to transform literature into a multimodal reading experience. In this augmented literature, the text provides its reader with verbal information, while other senses perceive context through non-verbal cues. We use our own framework to design this storytelling across senses integrating text, sound, animation and tactile sensation. The intention is to develop a multimodal storytelling that enhances the readers imagination, improving the reading experience. In this way, we believe, readers will completely immerse into the narrative and enjoy reading more.

Original languageEnglish
Title of host publicationUbiComp and ISWC 2015 - Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the Proceedings of the 2015 ACM International Symposium on Wearable Computers
PublisherAssociation for Computing Machinery, Inc
Pages353-356
Number of pages4
ISBN (Print)9781450335751
DOIs
Publication statusPublished - 2015 Sep 7
EventACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2015 ACM International Symposium on Wearable Computers, UbiComp and ISWC 2015 - Osaka, Japan
Duration: 2015 Sep 72015 Sep 11

Other

OtherACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2015 ACM International Symposium on Wearable Computers, UbiComp and ISWC 2015
CountryJapan
CityOsaka
Period15/9/715/9/11

Fingerprint

Animation
Transducers
Acoustic waves
Feedback

Keywords

  • Dual-coding
  • Multimodal
  • Reading
  • Valid co-occurrences

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Software

Cite this

Gu, H., Sanchez, S., Kunze, K. S., & Inami, M. (2015). An augmented e-reader for multimodal literacy. In UbiComp and ISWC 2015 - Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the Proceedings of the 2015 ACM International Symposium on Wearable Computers (pp. 353-356). Association for Computing Machinery, Inc. https://doi.org/10.1145/2800835.2800897

An augmented e-reader for multimodal literacy. / Gu, Heng; Sanchez, Susana; Kunze, Kai Steven; Inami, Masahiko.

UbiComp and ISWC 2015 - Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the Proceedings of the 2015 ACM International Symposium on Wearable Computers. Association for Computing Machinery, Inc, 2015. p. 353-356.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Gu, H, Sanchez, S, Kunze, KS & Inami, M 2015, An augmented e-reader for multimodal literacy. in UbiComp and ISWC 2015 - Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the Proceedings of the 2015 ACM International Symposium on Wearable Computers. Association for Computing Machinery, Inc, pp. 353-356, ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2015 ACM International Symposium on Wearable Computers, UbiComp and ISWC 2015, Osaka, Japan, 15/9/7. https://doi.org/10.1145/2800835.2800897
Gu H, Sanchez S, Kunze KS, Inami M. An augmented e-reader for multimodal literacy. In UbiComp and ISWC 2015 - Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the Proceedings of the 2015 ACM International Symposium on Wearable Computers. Association for Computing Machinery, Inc. 2015. p. 353-356 https://doi.org/10.1145/2800835.2800897
Gu, Heng ; Sanchez, Susana ; Kunze, Kai Steven ; Inami, Masahiko. / An augmented e-reader for multimodal literacy. UbiComp and ISWC 2015 - Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the Proceedings of the 2015 ACM International Symposium on Wearable Computers. Association for Computing Machinery, Inc, 2015. pp. 353-356
@inproceedings{822aa4e457dc4535bf38171cf341b43a,
title = "An augmented e-reader for multimodal literacy",
abstract = "We present an initial prototype to extend a reading tablet for haptic feedback -with a surface vibration transducer. This, in order to transform literature into a multimodal reading experience. In this augmented literature, the text provides its reader with verbal information, while other senses perceive context through non-verbal cues. We use our own framework to design this storytelling across senses integrating text, sound, animation and tactile sensation. The intention is to develop a multimodal storytelling that enhances the readers imagination, improving the reading experience. In this way, we believe, readers will completely immerse into the narrative and enjoy reading more.",
keywords = "Dual-coding, Multimodal, Reading, Valid co-occurrences",
author = "Heng Gu and Susana Sanchez and Kunze, {Kai Steven} and Masahiko Inami",
year = "2015",
month = "9",
day = "7",
doi = "10.1145/2800835.2800897",
language = "English",
isbn = "9781450335751",
pages = "353--356",
booktitle = "UbiComp and ISWC 2015 - Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the Proceedings of the 2015 ACM International Symposium on Wearable Computers",
publisher = "Association for Computing Machinery, Inc",

}

TY - GEN

T1 - An augmented e-reader for multimodal literacy

AU - Gu, Heng

AU - Sanchez, Susana

AU - Kunze, Kai Steven

AU - Inami, Masahiko

PY - 2015/9/7

Y1 - 2015/9/7

N2 - We present an initial prototype to extend a reading tablet for haptic feedback -with a surface vibration transducer. This, in order to transform literature into a multimodal reading experience. In this augmented literature, the text provides its reader with verbal information, while other senses perceive context through non-verbal cues. We use our own framework to design this storytelling across senses integrating text, sound, animation and tactile sensation. The intention is to develop a multimodal storytelling that enhances the readers imagination, improving the reading experience. In this way, we believe, readers will completely immerse into the narrative and enjoy reading more.

AB - We present an initial prototype to extend a reading tablet for haptic feedback -with a surface vibration transducer. This, in order to transform literature into a multimodal reading experience. In this augmented literature, the text provides its reader with verbal information, while other senses perceive context through non-verbal cues. We use our own framework to design this storytelling across senses integrating text, sound, animation and tactile sensation. The intention is to develop a multimodal storytelling that enhances the readers imagination, improving the reading experience. In this way, we believe, readers will completely immerse into the narrative and enjoy reading more.

KW - Dual-coding

KW - Multimodal

KW - Reading

KW - Valid co-occurrences

UR - http://www.scopus.com/inward/record.url?scp=84962551162&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84962551162&partnerID=8YFLogxK

U2 - 10.1145/2800835.2800897

DO - 10.1145/2800835.2800897

M3 - Conference contribution

AN - SCOPUS:84962551162

SN - 9781450335751

SP - 353

EP - 356

BT - UbiComp and ISWC 2015 - Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the Proceedings of the 2015 ACM International Symposium on Wearable Computers

PB - Association for Computing Machinery, Inc

ER -