TY - GEN
T1 - Multimodal literacy
T2 - ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2015 ACM International Symposium on Wearable Computers, UbiComp and ISWC 2015
AU - Sanchez, Suana
AU - Gu, Heng
AU - Kunze, Kai
AU - Inami, Masahiko
N1 - Funding Information:
This work is supported in part by the CREST project from JST and JSPS Grant-in-Aid Young Researchers B No. 26730095.
PY - 2015/9/7
Y1 - 2015/9/7
N2 - This paper describes a novel framework for literature that presents the reader with a multimodal experience (visual, haptic and auditory). We present an initial prototype, ex- tending an iPad with a surface vibration transducer for hap- tic feedback to augment the reading of a short story. Text provides the reader with verbal information, whereas other senses perceive the context through non-verbal cues. We use our own framework to create this storytelling across senses, integrating text, sound, animation and tactile sen- sation. We believe with a multi-modal reading experience, users will enjoy reading more, will feel more immersed and remember the narrative better.
AB - This paper describes a novel framework for literature that presents the reader with a multimodal experience (visual, haptic and auditory). We present an initial prototype, ex- tending an iPad with a surface vibration transducer for hap- tic feedback to augment the reading of a short story. Text provides the reader with verbal information, whereas other senses perceive the context through non-verbal cues. We use our own framework to create this storytelling across senses, integrating text, sound, animation and tactile sen- sation. We believe with a multi-modal reading experience, users will enjoy reading more, will feel more immersed and remember the narrative better.
KW - Dual-coding
KW - Multimodal
KW - Reading
KW - Valid co-occurrences
UR - http://www.scopus.com/inward/record.url?scp=84962538791&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84962538791&partnerID=8YFLogxK
U2 - 10.1145/2800835.2807940
DO - 10.1145/2800835.2807940
M3 - Conference contribution
AN - SCOPUS:84962538791
T3 - UbiComp and ISWC 2015 - Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the Proceedings of the 2015 ACM International Symposium on Wearable Computers
SP - 1257
EP - 1260
BT - UbiComp and ISWC 2015 - Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the Proceedings of the 2015 ACM International Symposium on Wearable Computers
PB - Association for Computing Machinery, Inc
Y2 - 7 September 2015 through 11 September 2015
ER -