A cross-media music retrieval system by converting color changes into tonal changes

Yoshiyuki Kato, Shuichi Kurabayashi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose a cross-media music retrieval system that provides its users with a toolkit for formulating their own query by describing their emotional requirements for changes in mood of music as a sequence of images. To interpret the query, the system uses a metric space to convert the color changes in images into continuous tonal changes in music, and vice versa. The system provides delta functions for music and images to compute the changes as distance values in the metric space. Our system calculates the sentiment-oriented relevance score of the query and music by comparing their computed distance values. The advantage of the system is that it provides a bridge between heterogeneous image and music criteria by converting the visual impressions of images into a time-oriented invisible impression of music. This bridging operation is the foundation of our visual query construction method. This method allows users to search for music by subtly manipulating the form of a query in a trial-and-error manner by detecting the moods of music as color changes in images. This method is suitable for searching for unknown music that satisfies the preferences of users in web-based music resources that store many types of music, but without any publishing information related to the music.

Original languageEnglish
Title of host publicationProceedings - IEEE International Conference on Mobile Data Management
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages349-352
Number of pages4
Volume1
ISBN (Print)9781479957057
DOIs
Publication statusPublished - 2014 Oct 5
Event15th IEEE International Conference on Mobile Data Management, IEEE MDM 2014 - Brisbane, Australia
Duration: 2014 Jul 152014 Jul 18

Other

Other15th IEEE International Conference on Mobile Data Management, IEEE MDM 2014
CountryAustralia
CityBrisbane
Period14/7/1514/7/18

Fingerprint

Color
Delta functions

Keywords

  • Media Retrieval
  • Mood Analysis
  • Multimedia Database
  • User Interface

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Kato, Y., & Kurabayashi, S. (2014). A cross-media music retrieval system by converting color changes into tonal changes. In Proceedings - IEEE International Conference on Mobile Data Management (Vol. 1, pp. 349-352). [6916943] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MDM.2014.51

A cross-media music retrieval system by converting color changes into tonal changes. / Kato, Yoshiyuki; Kurabayashi, Shuichi.

Proceedings - IEEE International Conference on Mobile Data Management. Vol. 1 Institute of Electrical and Electronics Engineers Inc., 2014. p. 349-352 6916943.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kato, Y & Kurabayashi, S 2014, A cross-media music retrieval system by converting color changes into tonal changes. in Proceedings - IEEE International Conference on Mobile Data Management. vol. 1, 6916943, Institute of Electrical and Electronics Engineers Inc., pp. 349-352, 15th IEEE International Conference on Mobile Data Management, IEEE MDM 2014, Brisbane, Australia, 14/7/15. https://doi.org/10.1109/MDM.2014.51
Kato Y, Kurabayashi S. A cross-media music retrieval system by converting color changes into tonal changes. In Proceedings - IEEE International Conference on Mobile Data Management. Vol. 1. Institute of Electrical and Electronics Engineers Inc. 2014. p. 349-352. 6916943 https://doi.org/10.1109/MDM.2014.51
Kato, Yoshiyuki ; Kurabayashi, Shuichi. / A cross-media music retrieval system by converting color changes into tonal changes. Proceedings - IEEE International Conference on Mobile Data Management. Vol. 1 Institute of Electrical and Electronics Engineers Inc., 2014. pp. 349-352
@inproceedings{83c61cf2322646f388526d9a7d8e32aa,
title = "A cross-media music retrieval system by converting color changes into tonal changes",
abstract = "We propose a cross-media music retrieval system that provides its users with a toolkit for formulating their own query by describing their emotional requirements for changes in mood of music as a sequence of images. To interpret the query, the system uses a metric space to convert the color changes in images into continuous tonal changes in music, and vice versa. The system provides delta functions for music and images to compute the changes as distance values in the metric space. Our system calculates the sentiment-oriented relevance score of the query and music by comparing their computed distance values. The advantage of the system is that it provides a bridge between heterogeneous image and music criteria by converting the visual impressions of images into a time-oriented invisible impression of music. This bridging operation is the foundation of our visual query construction method. This method allows users to search for music by subtly manipulating the form of a query in a trial-and-error manner by detecting the moods of music as color changes in images. This method is suitable for searching for unknown music that satisfies the preferences of users in web-based music resources that store many types of music, but without any publishing information related to the music.",
keywords = "Media Retrieval, Mood Analysis, Multimedia Database, User Interface",
author = "Yoshiyuki Kato and Shuichi Kurabayashi",
year = "2014",
month = "10",
day = "5",
doi = "10.1109/MDM.2014.51",
language = "English",
isbn = "9781479957057",
volume = "1",
pages = "349--352",
booktitle = "Proceedings - IEEE International Conference on Mobile Data Management",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - A cross-media music retrieval system by converting color changes into tonal changes

AU - Kato, Yoshiyuki

AU - Kurabayashi, Shuichi

PY - 2014/10/5

Y1 - 2014/10/5

N2 - We propose a cross-media music retrieval system that provides its users with a toolkit for formulating their own query by describing their emotional requirements for changes in mood of music as a sequence of images. To interpret the query, the system uses a metric space to convert the color changes in images into continuous tonal changes in music, and vice versa. The system provides delta functions for music and images to compute the changes as distance values in the metric space. Our system calculates the sentiment-oriented relevance score of the query and music by comparing their computed distance values. The advantage of the system is that it provides a bridge between heterogeneous image and music criteria by converting the visual impressions of images into a time-oriented invisible impression of music. This bridging operation is the foundation of our visual query construction method. This method allows users to search for music by subtly manipulating the form of a query in a trial-and-error manner by detecting the moods of music as color changes in images. This method is suitable for searching for unknown music that satisfies the preferences of users in web-based music resources that store many types of music, but without any publishing information related to the music.

AB - We propose a cross-media music retrieval system that provides its users with a toolkit for formulating their own query by describing their emotional requirements for changes in mood of music as a sequence of images. To interpret the query, the system uses a metric space to convert the color changes in images into continuous tonal changes in music, and vice versa. The system provides delta functions for music and images to compute the changes as distance values in the metric space. Our system calculates the sentiment-oriented relevance score of the query and music by comparing their computed distance values. The advantage of the system is that it provides a bridge between heterogeneous image and music criteria by converting the visual impressions of images into a time-oriented invisible impression of music. This bridging operation is the foundation of our visual query construction method. This method allows users to search for music by subtly manipulating the form of a query in a trial-and-error manner by detecting the moods of music as color changes in images. This method is suitable for searching for unknown music that satisfies the preferences of users in web-based music resources that store many types of music, but without any publishing information related to the music.

KW - Media Retrieval

KW - Mood Analysis

KW - Multimedia Database

KW - User Interface

UR - http://www.scopus.com/inward/record.url?scp=84907977754&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84907977754&partnerID=8YFLogxK

U2 - 10.1109/MDM.2014.51

DO - 10.1109/MDM.2014.51

M3 - Conference contribution

AN - SCOPUS:84907977754

SN - 9781479957057

VL - 1

SP - 349

EP - 352

BT - Proceedings - IEEE International Conference on Mobile Data Management

PB - Institute of Electrical and Electronics Engineers Inc.

ER -