Difference between revisions of "2006:Audio Cover Song Identification Results"
| Line 2: | Line 2: | ||
==Introduction== | ==Introduction== | ||
These are the results for the 2006 running of the Audio Cover Song Identification task. For background information about this task set please refer to the [[Audio Cover Song]] page. | These are the results for the 2006 running of the Audio Cover Song Identification task. For background information about this task set please refer to the [[Audio Cover Song]] page. | ||
| + | Each system was given 1000 songs which consists of 30 different class of cover songs where each cover song class was represented by 11 different versions. This 330 cover songs were used as queries and systems were required to return 10 results for each query. | ||
| + | |||
===General Legend=== | ===General Legend=== | ||
Revision as of 17:27, 3 October 2006
Contents
Introduction
These are the results for the 2006 running of the Audio Cover Song Identification task. For background information about this task set please refer to the Audio Cover Song page. Each system was given 1000 songs which consists of 30 different class of cover songs where each cover song class was represented by 11 different versions. This 330 cover songs were used as queries and systems were required to return 10 results for each query.
General Legend
Team ID
CS = Christian Sailer (1)
DE = Daniel P. W. Ellis (1)
KL1 = Kyogu Lee 1 (1)
KL2 = Kyogu Lee 2 (1)
KWL = Kris West (Likely) (2)
KWT = Kris West (Trans) (2)
LR = Thomas Lidy and Andreas Rauber (2)
TP = Tim Pohle (2)
(1) Denotes submissions specifically designed to detect cover song variants.
(2) Denotes submissions not-specifically designed to detect cover song variants (See Audio Music Similarity and Retrieval Results).
Calculating Summary Measures
MRR = Mean Reciprocal Rank. Reciprocal rank is the reciprocal of the rank of the first correctly identified cover for each query (1/rank). These values are averaged for each cover song group as well as overall.
Overall Summary Results
file /nema-raid/www/mirex/results/coversong_overall.csv not found
Audio Cover Song Identification Runtime Data
file /nema-raid/www/mirex/results/ac06_runtime.csv not found
Friedman Test with Multiple Comparisons Results (p=0.05)
The Friedman test was run in MATLAB against the MRR summary data over the 30 song groups.
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
file /nema-raid/www/mirex/results/cover_song_sum_friedman.csv not found file /nema-raid/www/mirex/results/audiocover_friedman_mrr.csv not found
Mean Reciprocal Rank (MRR) Results
file /nema-raid/www/mirex/results/coversong_mrr.csv not found
Raw Results
The total number of correctly identified covers for each query, and the maximum number of covers identified for each cover song group. file /nema-raid/www/mirex/results/coversong_results.csv not found
