An Affective Video Generation System Supporting Impromptu Musical Performance

Anri Kobayashi, Issei Fujishiro

研究成果: Conference contribution

2 被引用数 (Scopus)


When a musical instrument player performs music, the accompanying visual information can have a significant effect on the performance. For example, several players in a jam session may change their style of playing immediately by closely examining the co-players' expressions and behaviors and predicting their emotion and intention on the fly. In this paper, we propose a system that generates videos in response to the impromptu performance of a single musical instrumental player. The system evaluates the input signals in an affective way and generates a corresponding video based on the results of the evaluation. The player tends to change his/her performance while being inspired by the generated video and giving further triggers for the system to modify the video. The system aims to establish such an affective loop, where it is expected to act as a «co-player» who continues having an influence on the performance of the real player. The final goal of this study is to improve the quality of the player's performing experience through such interactions between the player and the system. By conducting a user evaluation, it was proven that we were able to inspire an amateur guitarist as the subject through the affective video generation and provide a cyberworld where he is allowed to experience a better performance than playing alone.

ホスト出版物のタイトルProceedings - 2016 International Conference on Cyberworlds, CW 2016
出版社Institute of Electrical and Electronics Engineers Inc.
出版ステータスPublished - 2016 11月 23
イベント2016 International Conference on Cyberworlds, CW 2016 - Chongqing, China
継続期間: 2016 9月 282016 9月 30


Other2016 International Conference on Cyberworlds, CW 2016

ASJC Scopus subject areas

  • 人工知能
  • コンピュータ ネットワークおよび通信
  • コンピュータ サイエンスの応用
  • コンピュータ ビジョンおよびパターン認識
  • 人間とコンピュータの相互作用
  • 信号処理


「An Affective Video Generation System Supporting Impromptu Musical Performance」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。