Investigating Textual Visual Sound Effects in a Virtual Environment and their impacts on Object Perception and Sound Perception

Thibault Fabre, Adrien Verhulst, Alfonso Balandra, Maki Sugimoto, Masahiko Inami

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In comics, Textual Sound Effects (TE) can describe sounds, but also actions, events, etc. TE could be used in Virtual Environment to efficiently create an easily recognizable scene and add more information to objects at a relatively low design cost. We investigate the impact of TE in a Virtual Environment on objects’ material perception (on category and properties) and on sound perception (on volume [dB] and spatial position). Participants (N=13, repeated measures) categorized metallic and wooden spheres and significantly changed their reaction time depending on the TE congruence with the spheres’ material/sound. They then rated a sphere’s properties (i.e., wetness, warmness, softness, smoothness, and dullness) and significantly changed their rating depending on the TE. When comparing 2 sound volumes, they perceived a sound associated with a shrinking TE as less loud and a sound associated with a growing TE as louder. When locating an audio source location, they located it significantly closer to a TE.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2021
EditorsMaud Marchal, Jonathan Ventura, Anne-Helene Olivier, Lili Wang, Rafael Radkowski
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages320-328
Number of pages9
ISBN (Electronic)9781665401586
DOIs
Publication statusPublished - 2021
Event20th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2021 - Virtual, Online, Italy
Duration: 2021 Oct 42021 Oct 8

Publication series

NameProceedings - 2021 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2021

Conference

Conference20th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2021
Country/TerritoryItaly
CityVirtual, Online
Period21/10/421/10/8

Keywords

  • Human computer interaction (HCI)
  • Human-centered computing
  • Interaction paradigms
  • Virtual reality

ASJC Scopus subject areas

  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Investigating Textual Visual Sound Effects in a Virtual Environment and their impacts on Object Perception and Sound Perception'. Together they form a unique fingerprint.

Cite this