2006:Symbolic Melodic Similarity

Task suggestion: Symbolic Melodic Similarity

We are willing to organize another symbolic melodic similarity task.

Proposed tasks

1. Retrieve the most similar incipits from the UK subset of the RISM A/II collection, given one of the incipits as a query, and rank them by melodic similarity. Both the query and the collection are monophonic.

2. Like task 1, but with a collection of polyphonic MIDI files to be searched for matches. The query would still be monophonic.

We propose to do things similarly to last year's symbolic melodic similarity contest, but change the following things:

- use a larger dataset. Instead of 589 incipits, use the UK subset of the RISM collection. That dataset is available, and RISM UK is happy with us using it for this purpose. It contains 35,000 records (some of the records have no incipits, some have multiple incipits, so the number of incipits could be somewhat different from 35,000).

- add another task: use polyphonic MIDI files. For example, we could use Alexa Web Search for harvesting up to 110,000 MIDI files from the World Wide Web.

Some points for discussion

- We would need to establish ground truths for both data sets.

- For the RISM dataset, we already have some software that facilitates creating a ground truth: a filtering script that can help with the task of selecting approximately 50 candidates for matches for each query (to be presented to human experts), and another web-based script for collecting human experts' opinions on what an ideal search result would be, and for consolidating those opinions.

- Selecting 50 candidates out of 35,000 to be shown to human experts is error-prone - we might well miss some incipits that would be true positives. We need to decide before the competition how we are going to correct such problems that might show up only when some submitted algorithm finds an incipit that should reasonably be classified as a true positive but was accidentally excluded in the filtering process for the ground truth.

- For the not yet existing polyphonic dataset, we also don't have any ground truth-related tools yet.

- Establishing a ground truth is a lot of work, and it would be desirable to split it in some way, for example among the participants of the competition.

Please share your thoughts!

Best regards,

Anna Pienimäki and Rainer Typke