Abstract
When a musical instrument player performs music, the accompanying visual information can have a significant effect on the performance. For example, several players in a jam session may change their style of playing immediately by closely examining the co-players' expressions and behaviors and predicting their emotion and intention on the fly. In this paper, we propose a system that generates videos in response to the impromptu performance of a single musical instrumental player. The system evaluates the input signals in an affective way and generates a corresponding video based on the results of the evaluation. The player tends to change his/her performance while being inspired by the generated video and giving further triggers for the system to modify the video. The system aims to establish such an affective loop, where it is expected to act as a «co-player» who continues having an influence on the performance of the real player. The final goal of this study is to improve the quality of the player's performing experience through such interactions between the player and the system. By conducting a user evaluation, it was proven that we were able to inspire an amateur guitarist as the subject through the affective video generation and provide a cyberworld where he is allowed to experience a better performance than playing alone.
| Original language | English |
|---|---|
| Title of host publication | Proceedings - 2016 International Conference on Cyberworlds, CW 2016 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 17-24 |
| Number of pages | 8 |
| ISBN (Electronic) | 9781509023035 |
| DOIs | |
| Publication status | Published - 2016 Nov 23 |
| Event | 2016 International Conference on Cyberworlds, CW 2016 - Chongqing, China Duration: 2016 Sep 28 → 2016 Sep 30 |
Other
| Other | 2016 International Conference on Cyberworlds, CW 2016 |
|---|---|
| Country | China |
| City | Chongqing |
| Period | 16/9/28 → 16/9/30 |
Fingerprint
Keywords
- Affective computing
- cell automata
- classifier
- music visualization
- musical performance
- video generation
ASJC Scopus subject areas
- Artificial Intelligence
- Computer Networks and Communications
- Computer Science Applications
- Computer Vision and Pattern Recognition
- Human-Computer Interaction
- Signal Processing
Cite this
An Affective Video Generation System Supporting Impromptu Musical Performance. / Kobayashi, Anri; Fujishiro, Issei.
Proceedings - 2016 International Conference on Cyberworlds, CW 2016. Institute of Electrical and Electronics Engineers Inc., 2016. p. 17-24 7756122.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - An Affective Video Generation System Supporting Impromptu Musical Performance
AU - Kobayashi, Anri
AU - Fujishiro, Issei
PY - 2016/11/23
Y1 - 2016/11/23
N2 - When a musical instrument player performs music, the accompanying visual information can have a significant effect on the performance. For example, several players in a jam session may change their style of playing immediately by closely examining the co-players' expressions and behaviors and predicting their emotion and intention on the fly. In this paper, we propose a system that generates videos in response to the impromptu performance of a single musical instrumental player. The system evaluates the input signals in an affective way and generates a corresponding video based on the results of the evaluation. The player tends to change his/her performance while being inspired by the generated video and giving further triggers for the system to modify the video. The system aims to establish such an affective loop, where it is expected to act as a «co-player» who continues having an influence on the performance of the real player. The final goal of this study is to improve the quality of the player's performing experience through such interactions between the player and the system. By conducting a user evaluation, it was proven that we were able to inspire an amateur guitarist as the subject through the affective video generation and provide a cyberworld where he is allowed to experience a better performance than playing alone.
AB - When a musical instrument player performs music, the accompanying visual information can have a significant effect on the performance. For example, several players in a jam session may change their style of playing immediately by closely examining the co-players' expressions and behaviors and predicting their emotion and intention on the fly. In this paper, we propose a system that generates videos in response to the impromptu performance of a single musical instrumental player. The system evaluates the input signals in an affective way and generates a corresponding video based on the results of the evaluation. The player tends to change his/her performance while being inspired by the generated video and giving further triggers for the system to modify the video. The system aims to establish such an affective loop, where it is expected to act as a «co-player» who continues having an influence on the performance of the real player. The final goal of this study is to improve the quality of the player's performing experience through such interactions between the player and the system. By conducting a user evaluation, it was proven that we were able to inspire an amateur guitarist as the subject through the affective video generation and provide a cyberworld where he is allowed to experience a better performance than playing alone.
KW - Affective computing
KW - cell automata
KW - classifier
KW - music visualization
KW - musical performance
KW - video generation
UR - http://www.scopus.com/inward/record.url?scp=85006850955&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85006850955&partnerID=8YFLogxK
U2 - 10.1109/CW.2016.11
DO - 10.1109/CW.2016.11
M3 - Conference contribution
AN - SCOPUS:85006850955
SP - 17
EP - 24
BT - Proceedings - 2016 International Conference on Cyberworlds, CW 2016
PB - Institute of Electrical and Electronics Engineers Inc.
ER -