Difference between revisions of "2007:Audio Music Similarity and Retrieval"
(→Data) |
m (→Data) |
||
| Line 19: | Line 19: | ||
* Is the proposal then to use ''exactly'' the same data as last year? I think this should be avoided if possible as it will highly bias toward algorithms which have been tuned to exploit last years data. Rather than (USPOP + USCRAP) or the old subset of Magnatune from 2004, perhaps a subset (random, or maybe artist filtered) of each combined with a subset from another CC repository (Jamendo perhaps) combined into a 5000ish song set for the task. | * Is the proposal then to use ''exactly'' the same data as last year? I think this should be avoided if possible as it will highly bias toward algorithms which have been tuned to exploit last years data. Rather than (USPOP + USCRAP) or the old subset of Magnatune from 2004, perhaps a subset (random, or maybe artist filtered) of each combined with a subset from another CC repository (Jamendo perhaps) combined into a 5000ish song set for the task. | ||
| − | + | -[[User:Bfields|Bfields]] 09:20, 31 July 2007 (CDT) | |
== Evaluation == | == Evaluation == | ||
Revision as of 08:22, 31 July 2007
Status
This is only a very basic draft version of a task proposal. Once more people show interest we can fill in the details.
Note that audio music similarity and retrieval algorithms have been evaluated at MIREX 2006.
Related MIREX 2007 task proposals:
- Audio Music Mood Classification
- Audio Artist Identification
- Audio Artist Similarity
- Audio Genre Classification
- Audio Cover song identification
Please feel free to edit this page.
Data
The data used for last year's audio similarity retrieval task (USPOP + USCRAP) could be used. In addition, the Magnatune data used for the ISMIR 2004 genre classification contest could be used.
Please edit this if you have suggestions to add or if you disagree.
- Is the proposal then to use exactly the same data as last year? I think this should be avoided if possible as it will highly bias toward algorithms which have been tuned to exploit last years data. Rather than (USPOP + USCRAP) or the old subset of Magnatune from 2004, perhaps a subset (random, or maybe artist filtered) of each combined with a subset from another CC repository (Jamendo perhaps) combined into a 5000ish song set for the task.
-Bfields 09:20, 31 July 2007 (CDT)
Evaluation
Similar procedures to the one used last year will be used.
As Magnatune and USPOP are freely available overfitting is possible. More interesting than the final ranking will be the accompanying papers in which the participants describe their work.
Please edit this if you have suggestions to add or if you disagree.
Audio format poll
<poll> Use the same 30 sec clips for analysis by participating algorithms as presented to human evaluators (necessary due to copyright restrictions)? Yes No, my algorithm needs longer clips No, I don't think this necessary No, for some other reason </poll>
<poll> What is your preferred audio format? Remember that the less audio data we have to process the larger the dataset can be... 22 khz mono WAV 22 khz stereo WAV 44 khz mono WAV 44 khz stereo WAV 22 khz mono MP3 128kb 22 khz stereo MP3 128kb 44 khz mono MP3 128kb 44 khz stereo MP3 128kb </poll>
Participants
If you think there is a slight chance that you might want to participate please add your name and email address here.
- Klaas Bosteels (firstname.lastname@gmail.com)
- Thomas Lidy (lastname@ifs.tuwien.ac.at)
- Elias Pampalk (firstname.lastname@gmail.com)
- Tim Pohle (firstname.lastname@jku.at)
- Kris West (kw at cmp dot uea dot ac dot uk)
- Julien Ricard (firstname.lastname@gmail.com)
- Abhinav Singh (abhinavs at iitg.ernet.in) and S.R.M.Prasanna (prasanna at iitg.ernet.in)
- Ben Fields (map01bf at gold dot ac dot uk)
- Christoph Bastuck (bsk at idmt.fhg.de)
- Aliaksandr Paradzinets (aliaksandr.paradzinets {at} ec-lyon.fr)
- Vitor Soares (firstname.lastname{at} clustermedialabs.com)
- Kai Chen('lastnamefirstname dot dr @gmail.com)
- Kurt Jacobson (firstname.lastname@elec.qmul.ac.uk)
