Abstract
When a musical instrument player performs music, the accompanying visual information can have a significant effect on the performance. For example, several players in a jam session may change their style of playing immediately by closely examining the co-players' expressions and behaviors and predicting their emotion and intention on the fly. In this paper, we propose a system that generates videos in response to the impromptu performance of a single musical instrumental player. The system evaluates the input signals in an affective way and generates a corresponding video based on the results of the evaluation. The player tends to change his/her performance while being inspired by the generated video and giving further triggers for the system to modify the video. The system aims to establish such an affective loop, where it is expected to act as a «co-player» who continues having an influence on the performance of the real player. The final goal of this study is to improve the quality of the player's performing experience through such interactions between the player and the system. By conducting a user evaluation, it was proven that we were able to inspire an amateur guitarist as the subject through the affective video generation and provide a cyberworld where he is allowed to experience a better performance than playing alone.
Original language | English |
---|---|
Title of host publication | Proceedings - 2016 International Conference on Cyberworlds, CW 2016 |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 17-24 |
Number of pages | 8 |
ISBN (Electronic) | 9781509023035 |
DOIs | |
Publication status | Published - 2016 Nov 23 |
Event | 2016 International Conference on Cyberworlds, CW 2016 - Chongqing, China Duration: 2016 Sep 28 → 2016 Sep 30 |
Other
Other | 2016 International Conference on Cyberworlds, CW 2016 |
---|---|
Country/Territory | China |
City | Chongqing |
Period | 16/9/28 → 16/9/30 |
Keywords
- Affective computing
- cell automata
- classifier
- music visualization
- musical performance
- video generation
ASJC Scopus subject areas
- Artificial Intelligence
- Computer Networks and Communications
- Computer Science Applications
- Computer Vision and Pattern Recognition
- Human-Computer Interaction
- Signal Processing