2012:Audio Music Similarity and Retrieval Results

From MIREX Wiki
Revision as of 14:30, 4 October 2012 by Kahyun Choi (talk | contribs) (→‎Friedman's Tests)

Introduction

These are the results for the 2011 running of the Audio Music Similarity and Retrieval task set. For background information about this task set please refer to the Audio Music Similarity and Retrieval page.

Each system was given 7000 songs chosen from IMIRSEL's "uspop", "uscrap" and "american" "classical" and "sundry" collections. Each system then returned a 7000x7000 distance matrix. 100 songs were randomly selected from the 10 genre groups (10 per genre) as queries and the first 5 most highly ranked songs out of the 7000 were extracted for each query (after filtering out the query itself, returned results from the same artist were also omitted). Then, for each query, the returned results (candidates) from all participants were grouped and were evaluated by human graders using the Evalutron 6000 grading system. Each individual query/candidate set was evaluated by a single grader. For each query/candidate pair, graders provided two scores. Graders were asked to provide 1 categorical BROAD score with 3 categories: NS,SS,VS as explained below, and one FINE score (in the range from 0 to 100). A description and analysis is provided below.

The systems read in 30 second audio clips as their raw data. The same 30 second clips were used in the grading stage.


General Legend

Team ID

Sub code Submission name Abstract Contributors
CTCP1 IrcamSimilarity_1 PDF Christophe Charbuillet, Geoffroy Peeters, Frédéric Cornu, Damien Tardieu
CTCP2 IrcamSimilarity_2 PDF Christophe Charbuillet, Geoffroy Peeters, Frédéric Cornu, Damien Tardieu
CTCP3 IrcamSimilarity_3 PDF Christophe Charbuillet, Geoffroy Peeters, Frédéric Cornu, Damien Tardieu
DM2 AMS_soton1 PDF Franz de Leon, Kirk Martinez
DM3 AMS_soton2 PDF Franz de Leon, Kirk Martinez
GKC1 RHYTHM_SIM PDF Aggelos Gkiokas, Vassilis Katsouros, George Carayannis
HKHLL1 MARG_SIM1 PDF Byeong-jun Han, Kyogu Lee, Ziwon Hyung, Hyunwoo Kim, Sheayun Lee
ML1 ML1 PDF Brian McFee, Gert Lanckriet
ML2 ML2 PDF Brian McFee, Gert Lanckriet
ML3 ML3 PDF Brian McFee, Gert Lanckriet
PS1 PS09 PDF Dominik Schnitzer, Tim Pohle
SSKS3 cbmr_sim PDF Klaus Seyerlehner, Markus Schedl, Peter Knees, Reinhard Sonnleitner
SSPK2 cbmr_sim_2010 PDF Klaus Seyerlehner, Markus Schedl, Peter Knees, Tim Pohle
STBD1 AMS2011_vario PDF Simone Sammartino, Cristina de la Bandera, Lorenzo J. Tardón, Isabel Barbancho
STBD2 AMS2011_multi PDF Simone Sammartino, Cristina de la Bandera, Lorenzo J. Tardón, Isabel Barbancho
STBD3 AMS2011_f0delta PDF Simone Sammartino, Cristina de la Bandera, Lorenzo J. Tardón, Isabel Barbancho
YL1 SC Learning PDF Yin-Tzu Lin, Wen-Huang Cheng, Ja-Ling Wu
ZYC2 SUMMs for AMS PDF Dongying Zhang, Deshun Yang, Xiaoou Chen

Broad Categories

NS = Not Similar
SS = Somewhat Similar
VS = Very Similar

Understanding Summary Measures

Fine = Has a range from 0 (failure) to 100 (perfection).
Broad = Has a range from 0 (failure) to 2 (perfection) as each query/candidate pair is scored with either NS=0, SS=1 or VS=2.

Human Evaluation

Overall Summary Results

Measure DM6 DM7 GT3 JR2 NHHL1 NHHL2 PS1 RW4 SSKP1 SSKS2
Average Fine Score 36.176 36.332 44.872 47.020 45.944 45.944 53.136 50.000 52.640 53.188
Average Cat Score 0.680 0.682 0.894 0.956 0.926 0.926 1.128 1.048 1.138 1.132

download these results as csv


Note:RZ1 is the random result for comparing purpose.

Friedman's Tests

Friedman's Test (FINE Scores)

The Friedman test was run in MATLAB against the Fine summary data over the 100 queries.
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

TeamID TeamID Lowerbound Mean Upperbound Significance
SSKS2 PS1 -2.014 -0.110 1.794 FALSE
SSKS2 SSKP1 -1.684 0.220 2.124 FALSE
SSKS2 RW4 -1.284 0.620 2.524 FALSE
SSKS2 JR2 -0.164 1.740 3.644 FALSE
SSKS2 NHHL2 0.596 2.500 4.404 TRUE
SSKS2 NHHL1 0.596 2.500 4.404 TRUE
SSKS2 GT3 0.616 2.520 4.424 TRUE
SSKS2 DM7 2.726 4.630 6.534 TRUE
SSKS2 DM6 2.776 4.680 6.584 TRUE
PS1 SSKP1 -1.574 0.330 2.234 FALSE
PS1 RW4 -1.174 0.730 2.634 FALSE
PS1 JR2 -0.054 1.850 3.754 FALSE
PS1 NHHL2 0.706 2.610 4.514 TRUE
PS1 NHHL1 0.706 2.610 4.514 TRUE
PS1 GT3 0.726 2.630 4.534 TRUE
PS1 DM7 2.836 4.740 6.644 TRUE
PS1 DM6 2.886 4.790 6.694 TRUE
SSKP1 RW4 -1.504 0.400 2.304 FALSE
SSKP1 JR2 -0.384 1.520 3.424 FALSE
SSKP1 NHHL2 0.376 2.280 4.184 TRUE
SSKP1 NHHL1 0.376 2.280 4.184 TRUE
SSKP1 GT3 0.396 2.300 4.204 TRUE
SSKP1 DM7 2.506 4.410 6.314 TRUE
SSKP1 DM6 2.556 4.460 6.364 TRUE
RW4 JR2 -0.784 1.120 3.024 FALSE
RW4 NHHL2 -0.024 1.880 3.784 FALSE
RW4 NHHL1 -0.024 1.880 3.784 FALSE
RW4 GT3 -0.004 1.900 3.804 FALSE
RW4 DM7 2.106 4.010 5.914 TRUE
RW4 DM6 2.156 4.060 5.964 TRUE
JR2 NHHL2 -1.144 0.760 2.664 FALSE
JR2 NHHL1 -1.144 0.760 2.664 FALSE
JR2 GT3 -1.124 0.780 2.684 FALSE
JR2 DM7 0.986 2.890 4.794 TRUE
JR2 DM6 1.036 2.940 4.844 TRUE
NHHL2 NHHL1 -1.904 0.000 1.904 FALSE
NHHL2 GT3 -1.884 0.020 1.924 FALSE
NHHL2 DM7 0.226 2.130 4.034 TRUE
NHHL2 DM6 0.276 2.180 4.084 TRUE
NHHL1 GT3 -1.884 0.020 1.924 FALSE
NHHL1 DM7 0.226 2.130 4.034 TRUE
NHHL1 DM6 0.276 2.180 4.084 TRUE
GT3 DM7 0.206 2.110 4.014 TRUE
GT3 DM6 0.256 2.160 4.064 TRUE
DM7 DM6 -1.854 0.050 1.954 FALSE

download these results as csv

500px

Friedman's Test (BROAD Scores)

The Friedman test was run in MATLAB against the BROAD summary data over the 100 queries.
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

TeamID TeamID Lowerbound Mean Upperbound Significance
SSKP1 SSKS2 -2.052 -0.210 1.632 FALSE
SSKP1 PS1 -1.682 0.160 2.002 FALSE
SSKP1 RW4 -1.022 0.820 2.662 FALSE
SSKP1 JR2 -0.262 1.580 3.422 FALSE
SSKP1 NHHL2 0.488 2.330 4.172 TRUE
SSKP1 NHHL1 0.488 2.330 4.172 TRUE
SSKP1 GT3 0.388 2.230 4.072 TRUE
SSKP1 DM7 2.538 4.380 6.222 TRUE
SSKP1 DM6 2.538 4.380 6.222 TRUE
SSKS2 PS1 -1.472 0.370 2.212 FALSE
SSKS2 RW4 -0.812 1.030 2.872 FALSE
SSKS2 JR2 -0.052 1.790 3.632 FALSE
SSKS2 NHHL2 0.698 2.540 4.382 TRUE
SSKS2 NHHL1 0.698 2.540 4.382 TRUE
SSKS2 GT3 0.598 2.440 4.282 TRUE
SSKS2 DM7 2.748 4.590 6.432 TRUE
SSKS2 DM6 2.748 4.590 6.432 TRUE
PS1 RW4 -1.182 0.660 2.502 FALSE
PS1 JR2 -0.422 1.420 3.262 FALSE
PS1 NHHL2 0.328 2.170 4.012 TRUE
PS1 NHHL1 0.328 2.170 4.012 TRUE
PS1 GT3 0.228 2.070 3.912 TRUE
PS1 DM7 2.378 4.220 6.062 TRUE
PS1 DM6 2.378 4.220 6.062 TRUE
RW4 JR2 -1.082 0.760 2.602 FALSE
RW4 NHHL2 -0.332 1.510 3.352 FALSE
RW4 NHHL1 -0.332 1.510 3.352 FALSE
RW4 GT3 -0.432 1.410 3.252 FALSE
RW4 DM7 1.718 3.560 5.402 TRUE
RW4 DM6 1.718 3.560 5.402 TRUE
JR2 NHHL2 -1.092 0.750 2.592 FALSE
JR2 NHHL1 -1.092 0.750 2.592 FALSE
JR2 GT3 -1.192 0.650 2.492 FALSE
JR2 DM7 0.958 2.800 4.642 TRUE
JR2 DM6 0.958 2.800 4.642 TRUE
NHHL2 NHHL1 -1.842 0.000 1.842 FALSE
NHHL2 GT3 -1.942 -0.100 1.742 FALSE
NHHL2 DM7 0.208 2.050 3.892 TRUE
NHHL2 DM6 0.208 2.050 3.892 TRUE
NHHL1 GT3 -1.942 -0.100 1.742 FALSE
NHHL1 DM7 0.208 2.050 3.892 TRUE
NHHL1 DM6 0.208 2.050 3.892 TRUE
GT3 DM7 0.308 2.150 3.992 TRUE
GT3 DM6 0.308 2.150 3.992 TRUE
DM7 DM6 -1.842 0.000 1.842 FALSE

download these results as csv

500px

Summary Results by Query

FINE Scores

These are the mean FINE scores per query assigned by Evalutron graders. The FINE scores for the 5 candidates returned per algorithm, per query, have been averaged. Values are bounded between 0 and 100. A perfect score would be 100. Genre labels have been included for reference.

Genre Query CTCP1 CTCP2 CTCP3 DM2 DM3 GKC1 HKHLL1 ML1 ML2 ML3 PS1 SSKS3 SSPK2 STBD1 STBD2 STBD3 YL1 ZYC2
BAROQUE d006677 83.4 84.2 84.8 84.2 84.2 51.4 78.6 74.4 77.8 73.8 84.0 87.4 85.8 65.0 69.8 51.6 84.8 83.2
BAROQUE d006696 78.8 96.2 80.0 86.6 86.8 52.6 87.2 86.8 86.8 78.0 86.8 79.0 88.0 32.2 39.0 15.6 52.0 74.4
BAROQUE d007449 71.8 86.0 71.8 85.2 85.2 48.8 54.4 69.6 75.0 81.4 84.6 86.4 85.6 60.2 74.0 59.4 17.6 72.4
BAROQUE d008986 82.4 82.8 64.4 65.0 68.0 13.4 66.8 77.4 61.4 72.0 81.4 70.8 83.8 22.0 0.0 20.2 8.8 62.2
BAROQUE d012398 67.0 83.0 66.0 81.0 76.0 72.0 71.0 62.0 74.0 74.0 81.0 75.0 76.0 70.0 55.0 43.0 79.0 74.0
BAROQUE d013785 18.6 12.0 20.6 0.0 0.0 8.4 0.0 4.0 0.0 0.0 11.0 11.8 34.6 0.0 21.2 1.8 0.0 13.6
BAROQUE d014514 75.8 73.8 76.4 34.6 35.2 19.4 73.4 71.8 75.2 75.2 76.6 66.6 70.0 29.6 17.8 18.6 44.4 75.6
BAROQUE d014776 61.2 59.2 68.6 62.2 62.2 22.2 44.6 43.6 37.2 38.6 61.4 67.4 71.6 33.2 36.6 25.8 44.6 69.8
BAROQUE d015410 21.6 25.8 21.0 24.6 30.6 31.0 22.0 22.2 22.0 22.6 35.0 35.6 36.8 33.0 19.0 32.0 24.4 34.4
BAROQUE d016296 67.8 69.4 43.8 53.2 53.2 18.8 32.0 61.0 54.6 65.8 68.6 53.0 55.4 13.2 5.0 47.2 43.0 54.2
BLUES e003498 23.0 19.2 21.8 7.8 7.8 12.0 3.0 12.4 6.8 6.8 29.2 23.2 8.0 21.0 8.6 3.2 8.4 13.8
BLUES e006591 35.2 37.8 36.2 33.4 33.4 14.0 30.2 39.6 33.2 44.8 56.6 45.0 51.2 36.8 13.6 45.8 47.4 27.0
BLUES e006594 83.2 67.6 83.2 87.4 87.4 4.6 84.0 70.0 33.2 67.8 78.0 86.0 85.0 41.0 55.4 32.4 69.4 42.8
BLUES e007332 81.6 80.0 81.6 68.0 64.4 32.2 76.4 76.0 71.8 74.6 81.2 81.0 85.2 44.0 61.2 36.2 60.6 82.8
BLUES e010207 59.8 63.6 65.0 49.0 49.0 22.2 22.8 23.2 22.4 24.2 80.0 75.2 59.6 25.2 19.4 46.6 53.4 39.6
BLUES e010715 74.2 68.2 76.0 63.8 63.8 23.8 40.6 67.0 49.6 70.0 76.2 75.2 72.8 51.6 56.2 35.6 54.2 10.4
BLUES e012145 64.8 64.8 57.4 50.0 50.0 49.0 35.2 32.6 14.2 43.8 62.2 53.0 49.2 49.8 44.4 22.6 58.4 50.8
BLUES e012250 71.6 75.6 73.2 65.2 65.2 60.2 66.6 29.6 58.8 61.0 77.6 67.4 62.8 28.0 46.6 33.8 37.4 66.0
BLUES e012576 40.0 36.0 20.0 17.0 23.0 50.0 23.0 28.0 18.0 17.0 64.0 33.0 28.0 26.0 57.6 46.0 61.0 40.0
BLUES e015955 80.8 77.8 82.8 67.2 67.2 33.0 61.6 39.2 52.4 49.4 84.8 81.8 82.2 27.4 34.6 41.0 50.8 54.4
CLASSICAL d002664 29.6 39.8 35.2 30.0 30.0 16.0 20.8 48.6 44.0 31.6 34.6 33.6 33.0 3.6 2.0 1.0 37.6 16.4
CLASSICAL d008359 71.4 69.0 62.6 69.4 69.2 32.8 69.0 58.6 60.0 60.6 74.8 69.4 74.0 37.0 37.2 28.4 63.8 52.4
CLASSICAL d008847 68.0 64.2 66.6 69.8 66.8 55.4 64.6 62.4 59.6 53.4 71.0 72.6 71.2 44.6 13.4 55.2 21.6 66.4
CLASSICAL d009363 31.0 25.0 30.2 26.8 28.0 41.4 28.0 38.2 39.2 32.6 43.8 28.8 30.6 32.8 29.6 26.2 36.2 38.8
CLASSICAL d010273 84.0 82.0 83.8 81.0 78.0 37.2 78.6 83.0 82.2 83.0 83.6 82.6 65.0 65.6 15.6 49.2 79.8 78.8
CLASSICAL d012347 87.6 82.8 88.4 86.2 86.2 10.6 76.4 79.2 85.6 82.6 90.0 90.6 90.0 7.4 14.0 73.6 66.6 69.0
CLASSICAL d013474 64.4 74.6 73.8 75.8 75.8 67.4 64.2 68.8 69.2 62.8 71.6 69.8 77.6 39.2 33.8 44.8 48.6 61.2
CLASSICAL d013555 12.6 9.2 25.6 23.8 16.8 14.6 21.0 22.2 21.6 21.8 17.2 23.6 27.4 9.2 14.2 6.2 20.2 24.6
CLASSICAL d013745 79.8 81.4 79.0 62.0 62.0 16.0 72.8 73.4 76.6 75.6 81.6 71.4 72.0 59.0 23.2 48.4 63.4 72.6
CLASSICAL d019428 70.8 72.6 67.2 60.6 60.6 25.8 36.4 58.2 57.4 56.2 64.0 65.6 61.2 53.2 42.0 7.8 39.8 50.2
COUNTRY b007754 55.2 65.0 50.0 51.0 55.0 5.0 0.0 60.8 44.8 33.2 58.2 41.0 49.2 23.0 33.8 14.4 54.8 40.2
COUNTRY e000004 56.4 65.2 54.4 35.0 35.0 28.4 50.6 20.0 39.2 26.8 26.2 70.4 67.0 24.4 32.2 15.2 41.0 28.4
COUNTRY e005074 62.0 73.0 65.0 50.0 48.0 46.0 68.0 42.0 52.0 62.0 61.0 58.0 68.0 31.0 46.0 10.0 48.0 67.0
COUNTRY e006998 71.0 71.0 53.0 27.0 34.0 23.0 28.0 40.0 27.0 35.0 66.0 74.0 59.0 3.0 23.0 15.0 24.0 41.0
COUNTRY e008019 76.0 76.0 70.0 66.0 66.0 50.0 58.0 68.0 60.0 70.0 82.0 68.0 70.0 66.0 36.0 24.0 66.0 78.0
COUNTRY e010395 53.4 52.2 47.6 54.2 51.8 23.2 36.8 31.0 37.4 26.4 55.4 50.8 65.6 43.2 36.2 18.6 19.6 46.4
COUNTRY e012084 68.8 71.6 82.2 76.2 72.0 13.0 75.2 63.6 51.2 57.8 82.6 69.2 80.4 55.4 45.2 70.8 72.8 70.4
COUNTRY e014772 42.2 40.4 41.0 46.4 49.2 33.8 40.2 39.2 35.8 38.6 41.0 40.8 50.0 37.0 37.2 39.2 38.8 51.2
COUNTRY e019603 47.8 45.2 38.8 47.8 47.8 13.6 22.8 37.6 38.0 31.8 36.8 33.4 51.6 16.4 8.6 9.8 45.4 30.6
COUNTRY e019738 71.0 59.0 65.0 67.0 67.0 38.0 59.0 62.0 64.0 61.0 57.0 64.0 67.0 43.0 49.0 33.0 55.0 65.0
EDANCE a005135 54.0 57.0 57.0 27.0 30.0 41.0 19.0 2.2 31.0 53.0 41.0 39.0 43.0 54.0 32.0 28.0 46.0 0.0
EDANCE a005589 54.4 37.4 32.4 22.6 22.6 22.0 0.0 28.0 13.8 44.2 33.6 26.8 39.8 3.4 21.0 0.0 45.0 35.2
EDANCE a007506 61.2 57.4 68.8 52.2 52.2 72.6 10.4 35.2 43.2 24.4 64.8 71.4 70.4 56.0 25.6 23.4 42.0 56.6
EDANCE b003709 44.4 42.8 45.2 50.6 50.6 38.6 5.8 36.6 37.6 38.8 45.6 44.2 57.6 29.6 18.0 30.6 21.0 12.0
EDANCE b005745 26.2 17.2 26.4 24.0 24.0 14.6 25.8 28.6 20.4 24.6 30.8 33.8 29.0 12.0 18.4 14.2 19.6 26.6
EDANCE b006254 35.6 28.6 49.0 1.6 3.4 44.6 15.0 4.6 9.4 5.2 18.4 54.4 49.0 1.2 10.0 6.4 32.0 2.4
EDANCE b011836 34.0 40.0 40.0 34.0 42.0 42.0 24.0 52.0 40.0 40.0 32.0 46.0 54.0 18.0 40.0 32.0 24.0 24.0
EDANCE b016602 66.0 66.0 63.0 33.4 46.6 57.0 49.8 34.6 59.6 42.0 49.2 70.6 55.0 30.4 30.4 49.2 45.6 37.4
EDANCE f003218 83.0 88.0 76.0 76.0 76.0 78.0 31.0 77.0 74.0 76.0 87.0 70.0 94.0 42.0 52.0 77.0 59.0 54.0
EDANCE f019182 32.4 53.0 28.4 42.8 30.2 35.6 15.6 16.8 18.6 23.0 34.6 51.6 26.4 24.4 29.6 17.2 65.0 34.4
JAZZ a006733 83.0 85.0 72.0 11.0 11.0 38.0 13.0 21.0 19.0 23.0 65.0 81.0 81.0 21.0 34.0 44.0 32.0 22.0
JAZZ a009161 61.0 57.8 64.4 33.0 29.2 25.6 45.8 60.6 56.2 53.6 40.0 55.6 76.0 33.4 21.2 12.0 63.6 57.4
JAZZ e004616 67.4 74.4 70.6 74.2 73.4 53.2 69.8 69.6 69.8 69.8 78.2 77.6 80.6 61.0 57.4 44.6 64.2 75.4
JAZZ e004656 79.0 82.6 74.2 70.2 70.2 29.0 64.8 73.4 78.4 73.4 72.4 79.0 81.0 51.0 47.2 40.2 69.8 76.4
JAZZ e008805 71.8 61.6 64.2 26.4 13.8 21.0 3.4 46.2 39.6 36.0 37.2 55.6 33.0 12.2 9.4 10.8 54.6 29.2
JAZZ e009003 87.0 83.0 83.0 74.0 74.0 45.0 45.0 58.0 77.0 74.0 82.0 85.0 83.0 41.0 57.0 64.0 10.4 87.0
JAZZ e009926 0.0 0.0 0.0 0.2 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 8.0 0.2 3.2 0.4 0.0
JAZZ e010314 87.2 90.0 83.0 16.0 19.0 33.8 27.8 41.2 53.2 42.6 59.4 78.8 76.4 32.6 24.4 33.4 12.6 62.2
JAZZ e010700 71.8 81.6 68.8 28.2 23.4 22.2 52.0 53.0 41.2 36.4 33.8 56.8 47.2 14.6 30.4 26.8 14.8 41.8
JAZZ e012958 6.0 8.0 0.0 23.2 23.2 20.4 0.0 0.0 6.0 0.0 20.6 0.6 0.0 15.0 2.0 0.0 0.0 0.0
METAL a000907 21.8 23.0 39.0 32.0 39.8 8.2 23.8 21.0 8.8 19.6 22.0 29.2 39.0 2.2 1.2 8.6 19.4 20.6
METAL b001064 72.2 74.2 71.4 49.4 44.8 34.8 44.0 45.4 67.8 71.4 46.4 79.8 71.4 20.4 25.6 49.8 66.8 44.8
METAL b005963 86.0 79.0 85.0 83.0 79.0 71.0 52.0 74.0 68.0 77.0 83.0 79.0 72.0 42.0 43.0 56.0 77.0 74.0
METAL b007455 59.6 68.2 57.4 57.2 63.8 11.6 48.2 55.0 55.8 50.0 65.6 41.2 65.8 36.2 24.0 11.4 48.4 61.6
METAL b008450 38.2 37.4 55.4 63.4 64.6 7.8 40.0 75.6 53.0 56.2 37.4 30.8 14.4 16.8 10.6 21.6 43.2 60.6
METAL b009939 35.0 27.0 32.4 17.0 15.0 15.8 17.0 48.0 46.0 33.4 40.0 41.0 46.0 18.2 14.0 25.2 23.0 40.0
METAL b012964 46.0 44.0 33.0 42.0 42.0 10.0 58.0 45.0 41.0 35.0 49.0 46.0 43.0 30.0 8.0 0.0 19.0 49.0
METAL b019036 33.8 44.2 49.2 26.0 33.2 25.8 29.6 34.0 44.4 43.0 35.2 41.8 29.0 36.2 36.8 20.0 27.0 17.4
METAL f007939 56.4 37.0 64.0 65.8 55.6 30.8 59.0 67.2 62.2 59.0 60.8 32.4 68.8 12.2 7.4 3.8 52.8 65.4
METAL f015638 73.0 74.8 81.4 64.0 64.0 4.2 52.4 61.2 61.2 59.8 77.0 83.2 90.2 19.8 15.6 31.6 68.0 67.4
RAPHIPHOP a000184 45.2 60.0 42.8 59.0 59.0 6.4 58.2 55.6 59.0 57.4 62.6 65.4 61.6 63.2 46.6 15.4 45.0 46.0
RAPHIPHOP a000369 72.6 78.4 78.0 68.8 53.8 22.4 46.2 72.4 77.4 73.2 79.2 79.2 82.2 73.8 74.0 68.8 75.2 82.0
RAPHIPHOP a002853 52.2 56.4 26.8 57.8 57.8 38.6 57.4 49.2 75.2 66.0 64.6 57.2 56.8 35.6 15.6 59.8 28.6 51.4
RAPHIPHOP a006408 73.6 66.0 68.6 72.6 71.8 56.4 73.8 76.6 71.8 70.2 70.8 76.4 71.4 69.8 68.4 73.2 73.8 74.0
RAPHIPHOP a007204 78.6 80.8 70.8 75.4 75.4 65.0 82.8 46.8 70.4 65.4 80.6 81.6 80.0 33.2 32.8 27.4 58.4 76.6
RAPHIPHOP a008101 68.4 73.2 63.8 57.8 51.0 11.0 49.8 69.2 63.2 66.4 71.2 71.0 68.4 45.8 31.2 55.0 54.8 68.2
RAPHIPHOP b001529 61.0 53.0 52.0 53.4 47.4 4.2 44.2 56.0 18.0 43.0 43.0 36.0 57.0 24.0 54.0 12.2 66.0 36.0
RAPHIPHOP b009504 57.0 59.0 50.0 67.0 67.0 13.2 54.4 56.4 63.0 74.2 61.6 43.6 48.8 6.2 3.0 57.4 60.6 60.2
RAPHIPHOP b012102 39.0 59.6 47.0 62.2 62.2 29.6 38.4 62.0 65.6 81.6 59.8 74.2 59.0 63.0 29.6 45.6 25.8 71.0
RAPHIPHOP b018710 41.6 69.0 48.0 71.6 64.2 26.6 32.4 59.0 67.2 59.0 66.0 69.4 77.0 26.0 32.2 65.2 42.8 46.0
ROCKROLL a003466 10.2 9.6 28.8 8.6 8.6 32.6 17.8 25.2 18.4 9.4 25.6 38.2 20.6 24.2 24.4 22.4 18.8 20.2
ROCKROLL b000979 29.0 41.0 28.0 38.0 50.0 0.0 23.0 49.0 36.0 30.0 44.0 63.0 37.0 18.0 14.0 8.0 16.0 58.0
ROCKROLL b001750 32.0 30.0 29.0 41.0 41.0 38.0 36.0 42.0 49.0 49.0 32.0 38.0 51.0 24.0 30.0 14.0 42.0 28.0
ROCKROLL b005580 41.2 35.2 35.8 31.6 31.6 27.2 22.0 41.0 27.4 18.6 32.6 33.8 36.4 34.4 24.0 2.6 23.8 36.8
ROCKROLL b008059 48.0 41.2 37.2 36.4 36.4 30.0 8.4 8.2 17.6 9.2 51.6 63.8 37.8 12.2 21.2 11.4 11.8 30.0
ROCKROLL b008990 48.4 70.4 46.2 44.2 44.2 38.8 25.4 41.6 29.0 38.0 54.6 64.6 46.0 32.2 24.0 1.6 43.4 51.2
ROCKROLL b011113 36.0 36.4 36.2 38.0 37.8 25.8 32.6 37.6 34.6 27.8 33.0 38.4 36.8 27.4 37.6 23.8 38.0 39.4
ROCKROLL b016165 66.8 69.8 60.4 57.6 59.2 49.8 54.4 46.6 29.0 45.4 61.0 59.2 49.8 31.4 12.6 30.0 57.8 73.6
ROCKROLL b017586 55.0 55.0 57.0 47.0 53.0 53.0 49.0 51.0 49.0 48.0 58.0 60.0 51.0 21.0 25.0 18.0 47.0 47.0
ROCKROLL b019928 49.8 50.2 55.2 59.4 55.4 29.2 31.8 44.2 40.0 50.2 49.2 50.0 52.2 37.4 51.4 42.2 43.8 53.8
ROMANTIC d005834 58.6 48.4 62.0 46.2 46.2 23.4 5.6 17.6 29.0 22.8 55.8 59.8 52.2 7.6 10.8 43.4 29.2 50.0
ROMANTIC d009949 67.0 64.8 59.8 65.2 65.2 50.6 59.8 62.0 62.0 62.0 67.4 68.4 65.4 58.4 47.4 20.6 35.6 69.4
ROMANTIC d010217 43.6 42.6 50.2 36.2 36.2 29.4 39.6 44.4 34.2 34.2 52.2 37.8 43.0 39.4 30.0 17.2 32.4 32.4
ROMANTIC d012214 66.0 68.4 61.0 65.2 64.0 55.0 53.2 55.6 58.4 55.2 63.8 66.0 70.6 34.0 43.0 46.0 45.0 69.2
ROMANTIC d015538 82.6 85.2 89.6 82.4 86.4 44.2 57.2 60.8 69.0 73.6 85.0 84.6 85.2 67.4 34.6 50.6 80.8 88.0
ROMANTIC d016565 46.0 54.0 45.0 33.0 37.0 25.0 24.0 22.0 29.0 18.0 48.0 42.0 47.0 54.0 29.0 7.0 15.0 41.0
ROMANTIC d017427 94.6 91.0 94.0 91.6 91.6 79.2 71.8 54.0 53.4 51.2 94.8 92.0 94.6 78.6 47.2 36.8 39.8 84.2
ROMANTIC d017784 69.0 74.0 69.0 73.0 73.0 68.0 71.0 47.0 66.0 51.0 75.0 67.0 77.0 87.0 63.0 85.0 33.0 73.0
ROMANTIC d018721 47.6 56.2 45.0 32.2 32.2 11.2 61.4 46.8 52.4 26.0 51.4 46.6 58.4 12.0 20.0 16.2 6.8 57.8
ROMANTIC d019573 73.2 77.4 75.0 65.0 67.6 16.2 51.4 66.4 50.8 74.2 75.0 72.0 72.6 51.4 26.0 27.6 36.8 65.0

download these results as csv

BROAD Scores

These are the mean BROAD scores per query assigned by Evalutron graders. The BROAD scores for the 5 candidates returned per algorithm, per query, have been averaged. Values are bounded between 0 (not similar) and 2 (very similar). A perfect score would be 2. Genre labels have been included for reference.

Genre Query CTCP1 CTCP2 CTCP3 DM2 DM3 GKC1 HKHLL1 ML1 ML2 ML3 PS1 SSKS3 SSPK2 STBD1 STBD2 STBD3 YL1 ZYC2
BAROQUE d006677 2.0 2.0 2.0 2.0 2.0 1.0 1.6 1.6 1.8 1.6 2.0 2.0 2.0 1.2 1.4 1.2 2.0 1.8
BAROQUE d006696 1.6 2.0 1.6 1.8 1.8 1.0 1.8 1.8 1.8 1.6 1.8 1.6 1.8 0.6 0.8 0.2 1.0 1.6
BAROQUE d007449 1.6 2.0 1.6 2.0 2.0 1.0 1.0 1.6 1.6 1.8 2.0 2.0 2.0 1.4 1.6 1.2 0.0 1.8
BAROQUE d008986 1.8 2.0 1.4 1.8 1.8 0.2 1.4 1.8 1.4 1.8 1.8 1.8 2.0 0.4 0.0 0.4 0.0 1.2
BAROQUE d012398 1.6 1.8 1.6 1.8 1.8 1.6 1.4 1.2 1.8 2.0 1.8 2.0 1.8 1.6 1.2 0.8 2.0 1.6
BAROQUE d013785 0.4 0.2 0.4 0.0 0.0 0.2 0.0 0.0 0.0 0.0 0.2 0.2 0.8 0.0 0.4 0.0 0.0 0.4
BAROQUE d014514 1.8 1.6 1.8 0.4 0.4 0.0 1.6 1.6 1.8 1.8 1.6 1.6 1.6 0.2 0.2 0.0 1.0 1.6
BAROQUE d014776 1.4 1.2 1.4 1.4 1.4 0.4 1.0 1.0 0.6 0.6 1.4 1.6 1.8 0.4 0.6 0.4 1.0 1.6
BAROQUE d015410 0.0 0.2 0.0 0.0 0.2 0.4 0.0 0.0 0.0 0.0 0.4 0.4 0.4 0.4 0.4 0.4 0.2 0.6
BAROQUE d016296 1.6 1.4 0.8 1.2 1.2 0.2 0.6 1.2 1.2 1.6 1.6 1.0 1.2 0.2 0.0 1.2 1.2 1.2
BLUES e003498 0.2 0.2 0.2 0.0 0.0 0.2 0.0 0.2 0.0 0.0 0.4 0.2 0.0 0.4 0.2 0.0 0.2 0.2
BLUES e006591 0.6 0.6 0.8 0.6 0.6 0.0 0.6 0.8 0.6 1.0 1.2 0.8 1.0 0.6 0.0 1.0 1.0 0.4
BLUES e006594 2.0 1.6 2.0 2.0 2.0 0.0 2.0 1.8 0.8 1.6 2.0 2.0 2.0 0.8 1.0 0.8 1.8 1.0
BLUES e007332 1.8 1.8 2.0 1.4 1.2 0.4 1.6 1.8 1.6 1.8 1.8 1.8 2.0 0.8 1.2 0.8 1.4 2.0
BLUES e010207 1.4 1.2 1.2 1.2 1.2 0.4 0.4 0.4 0.4 0.4 2.0 1.6 1.4 0.2 0.2 1.2 1.0 0.8
BLUES e010715 1.6 1.4 1.8 1.4 1.4 0.2 0.6 1.4 0.8 1.6 1.8 1.8 1.6 1.0 1.2 0.4 1.0 0.0
BLUES e012145 1.6 1.6 1.6 1.0 1.0 0.8 0.8 0.6 0.0 1.0 1.6 1.2 1.0 1.0 0.8 0.2 1.4 1.0
BLUES e012250 1.6 1.8 1.8 1.6 1.6 1.2 1.6 0.6 1.2 1.4 2.0 1.6 1.6 0.4 0.8 0.6 0.8 1.6
BLUES e012576 0.8 0.6 0.2 0.2 0.4 1.0 0.4 0.2 0.0 0.0 1.4 0.6 0.4 0.6 1.2 1.0 1.4 0.8
BLUES e015955 2.0 1.8 2.0 1.6 1.6 0.4 1.2 0.6 1.0 1.0 2.0 2.0 2.0 0.4 0.6 0.8 1.0 1.0
CLASSICAL d002664 0.6 1.0 0.8 0.6 0.6 0.4 0.4 1.0 1.0 0.4 0.8 0.6 0.6 0.0 0.0 0.0 0.8 0.2
CLASSICAL d008359 1.8 1.8 1.4 1.6 1.6 0.6 1.8 1.2 1.4 1.4 2.0 1.6 2.0 0.4 0.4 0.2 1.6 1.2
CLASSICAL d008847 1.2 1.0 1.2 1.4 1.4 1.0 1.0 1.0 1.0 0.8 1.0 1.2 1.2 0.6 0.4 1.0 0.2 1.0
CLASSICAL d009363 0.8 0.6 0.8 0.8 1.0 0.8 0.8 1.0 1.0 1.0 1.2 0.8 0.8 1.0 0.8 1.0 1.2 1.0
CLASSICAL d010273 2.0 2.0 2.0 2.0 1.8 0.8 1.8 2.0 2.0 2.0 2.0 2.0 1.6 1.4 0.0 1.0 2.0 2.0
CLASSICAL d012347 2.0 1.8 2.0 2.0 2.0 0.0 1.8 1.8 1.8 2.0 2.0 2.0 2.0 0.0 0.0 1.4 1.4 1.2
CLASSICAL d013474 1.6 2.0 1.8 2.0 2.0 1.6 1.6 1.8 1.6 1.2 1.8 1.8 2.0 0.8 0.8 1.0 1.2 1.6
CLASSICAL d013555 0.0 0.0 0.4 0.2 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.2 0.2 0.0 0.0 0.0 0.2 0.0
CLASSICAL d013745 2.0 2.0 2.0 1.6 1.6 0.2 2.0 2.0 1.8 2.0 2.0 2.0 2.0 1.6 0.4 1.2 1.6 1.8
CLASSICAL d019428 1.8 1.8 1.6 1.6 1.6 0.6 0.6 1.2 1.4 1.2 1.6 1.6 1.4 1.4 1.0 0.0 1.0 1.2
COUNTRY b007754 0.8 1.0 0.4 0.8 1.0 0.0 0.0 0.6 0.6 0.4 0.8 0.4 0.8 0.4 0.4 0.2 1.0 0.4
COUNTRY e000004 1.2 1.4 1.0 0.4 0.4 0.2 1.2 0.2 0.6 0.4 0.2 1.4 1.6 0.2 0.2 0.0 0.8 0.2
COUNTRY e005074 1.4 1.8 1.6 1.0 0.8 0.8 1.8 0.6 0.8 1.4 1.2 1.2 1.4 0.4 0.8 0.0 0.8 1.6
COUNTRY e006998 1.6 1.6 1.2 0.4 0.6 0.2 0.6 0.8 0.4 0.4 1.4 1.6 1.2 0.0 0.4 0.2 0.4 0.8
COUNTRY e008019 1.8 1.8 1.6 1.4 1.4 1.0 1.2 1.6 1.2 1.6 2.0 1.6 1.6 1.4 0.4 0.0 1.4 1.8
COUNTRY e010395 1.2 1.0 1.0 1.0 1.0 0.2 0.4 0.4 0.6 0.4 1.2 0.8 1.4 0.6 0.4 0.2 0.2 0.8
COUNTRY e012084 1.2 1.2 1.6 1.4 1.2 0.2 1.4 1.0 1.0 1.0 1.6 1.2 1.4 1.0 0.6 1.2 1.2 1.2
COUNTRY e014772 1.0 1.0 1.0 1.0 1.2 0.6 1.0 1.0 1.0 1.0 1.0 1.0 1.2 1.0 1.0 1.0 1.0 1.2
COUNTRY e019603 1.2 1.2 1.0 1.2 1.2 0.2 0.6 0.8 1.0 0.6 0.8 1.0 1.4 0.4 0.2 0.0 1.0 0.8
COUNTRY e019738 1.6 1.2 1.8 1.6 1.6 0.6 1.4 1.0 1.2 1.2 1.2 1.4 1.6 0.4 1.0 0.4 1.0 1.6
EDANCE a005135 1.0 1.2 1.2 0.4 0.4 0.8 0.2 0.0 0.4 1.2 0.6 0.8 1.0 1.4 0.8 0.4 1.0 0.0
EDANCE a005589 1.6 1.0 0.8 0.6 0.6 0.6 0.0 0.8 0.4 1.2 1.0 0.6 1.0 0.2 0.6 0.0 1.2 0.8
EDANCE a007506 1.2 1.0 1.4 0.8 0.8 1.6 0.0 0.4 0.6 0.0 1.0 1.2 1.4 1.0 0.4 0.0 0.6 1.0
EDANCE b003709 0.8 0.8 0.8 1.2 1.2 0.4 0.0 0.6 0.8 0.8 1.0 0.8 1.0 0.6 0.2 0.8 0.2 0.2
EDANCE b005745 0.6 0.4 0.6 0.6 0.6 0.2 0.6 0.6 0.4 0.6 0.8 0.8 0.6 0.0 0.2 0.2 0.4 0.6
EDANCE b006254 0.6 0.6 1.0 0.0 0.0 1.0 0.4 0.0 0.2 0.0 0.4 1.0 1.0 0.0 0.2 0.0 0.6 0.0
EDANCE b011836 0.6 0.8 0.8 0.6 0.8 0.8 0.2 1.0 0.6 0.6 0.4 1.0 1.2 0.0 0.8 0.4 0.2 0.2
EDANCE b016602 1.6 1.4 1.4 0.8 1.2 1.2 0.8 0.6 1.4 0.8 1.2 1.4 1.0 0.6 0.4 1.2 1.2 0.6
EDANCE f003218 1.8 1.8 1.8 1.8 1.8 1.8 0.8 1.6 1.8 1.8 2.0 1.6 2.0 0.6 1.2 1.6 1.2 1.0
EDANCE f019182 0.4 1.0 0.2 0.8 0.4 0.8 0.0 0.0 0.2 0.2 0.4 1.0 0.0 0.4 0.4 0.0 1.6 0.4
JAZZ a006733 2.0 2.0 1.6 0.0 0.0 0.6 0.0 0.0 0.0 0.2 1.4 2.0 2.0 0.2 0.6 1.0 0.4 0.2
JAZZ a009161 1.2 1.2 1.2 0.8 0.6 0.6 1.2 1.2 1.0 1.0 0.8 1.4 1.8 0.6 0.2 0.2 1.4 1.4
JAZZ e004616 1.4 1.6 1.6 1.8 1.8 1.2 1.6 1.6 1.6 1.6 1.8 2.0 2.0 1.4 1.4 1.0 1.4 2.0
JAZZ e004656 2.0 2.0 1.8 1.4 1.4 0.4 1.6 1.8 2.0 1.8 1.6 2.0 1.8 0.8 0.8 0.6 1.6 2.0
JAZZ e008805 1.6 1.4 1.4 0.4 0.0 0.4 0.0 1.0 1.0 0.8 1.0 1.2 0.8 0.0 0.2 0.2 1.0 0.4
JAZZ e009003 2.0 2.0 2.0 1.6 1.6 1.2 1.0 1.6 2.0 1.8 2.0 2.0 2.0 0.8 1.2 1.6 0.2 2.0
JAZZ e009926 0.0 0.0 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.2 0.0 0.0 0.2 0.0 0.0 0.0 0.0
JAZZ e010314 2.0 2.0 2.0 0.2 0.2 0.6 0.4 0.8 1.2 0.8 1.2 1.8 1.6 0.8 0.2 0.6 0.0 1.4
JAZZ e010700 1.6 1.8 1.4 0.2 0.2 0.4 1.0 1.2 0.6 0.6 0.6 1.4 1.0 0.0 0.2 0.4 0.0 0.6
JAZZ e012958 0.4 0.4 0.0 0.0 0.0 0.4 0.0 0.0 0.4 0.0 0.2 0.0 0.0 0.4 0.2 0.0 0.0 0.0
METAL a000907 0.2 0.4 0.6 0.4 0.6 0.2 0.4 0.2 0.0 0.2 0.2 0.4 0.6 0.0 0.0 0.2 0.2 0.4
METAL b001064 1.8 2.0 1.8 0.8 0.8 0.4 1.0 0.6 1.6 1.6 1.0 2.0 1.8 0.0 0.2 1.2 1.4 1.0
METAL b005963 1.8 1.8 2.0 1.8 1.6 1.8 1.2 1.6 1.4 1.8 2.0 1.8 1.4 0.8 1.0 1.4 1.6 1.8
METAL b007455 1.4 1.4 1.2 1.2 1.4 0.0 1.0 1.4 1.4 1.2 1.6 0.6 1.2 0.6 0.4 0.0 1.2 1.2
METAL b008450 1.2 1.2 1.4 1.6 1.6 0.4 0.8 1.8 1.6 1.6 0.8 1.0 0.6 0.2 0.4 0.6 1.2 1.8
METAL b009939 0.4 0.2 0.6 0.2 0.0 0.2 0.2 0.8 0.6 0.4 0.6 0.8 1.0 0.4 0.2 0.4 0.2 0.8
METAL b012964 0.8 0.6 0.4 0.8 0.8 0.2 1.2 1.0 0.6 0.6 0.8 0.8 0.8 0.4 0.2 0.0 0.2 1.0
METAL b019036 0.6 0.8 1.0 1.0 1.0 0.6 0.6 1.2 1.0 1.0 1.0 1.0 0.8 0.8 0.6 0.4 1.0 0.4
METAL f007939 1.2 0.8 1.4 1.2 1.0 0.6 1.4 1.4 1.4 1.2 1.2 0.6 1.4 0.2 0.0 0.0 1.0 1.4
METAL f015638 1.6 1.8 2.0 1.4 1.4 0.0 1.2 1.4 1.2 1.2 1.8 2.0 2.0 0.4 0.2 0.6 1.4 1.4
RAPHIPHOP a000184 1.0 1.4 0.8 1.4 1.4 0.0 1.4 1.4 1.4 1.4 1.8 1.8 1.6 1.8 1.2 0.2 1.0 1.2
RAPHIPHOP a000369 1.2 1.6 1.6 1.4 1.0 0.4 0.8 1.4 1.6 1.4 1.8 1.6 1.8 1.4 1.6 1.6 1.4 1.8
RAPHIPHOP a002853 1.2 1.4 0.4 1.4 1.4 0.8 1.6 1.2 2.0 1.6 1.6 1.6 1.6 0.8 0.2 1.4 0.6 1.0
RAPHIPHOP a006408 2.0 2.0 2.0 2.0 2.0 1.2 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 1.8 2.0 2.0 2.0
RAPHIPHOP a007204 1.8 1.8 1.4 2.0 2.0 1.4 2.0 0.8 1.6 1.4 1.8 2.0 1.8 0.6 0.6 0.2 1.4 2.0
RAPHIPHOP a008101 2.0 2.0 1.8 1.4 1.2 0.0 1.2 2.0 1.8 1.8 2.0 2.0 2.0 1.2 0.8 1.4 1.6 1.8
RAPHIPHOP b001529 1.4 1.2 1.0 1.2 1.0 0.0 0.8 1.2 0.2 0.8 0.8 0.8 1.4 0.2 1.2 0.2 1.4 0.8
RAPHIPHOP b009504 1.2 1.2 1.0 1.6 1.6 0.2 1.2 1.0 1.4 1.6 1.0 0.8 0.8 0.0 0.0 1.2 1.2 1.2
RAPHIPHOP b012102 0.8 1.2 1.0 1.4 1.4 0.6 0.8 1.2 1.4 2.0 1.4 1.8 1.2 1.4 0.8 0.8 0.6 1.6
RAPHIPHOP b018710 0.8 1.8 1.0 1.8 1.6 0.0 0.4 1.6 1.8 1.4 1.6 1.8 2.0 0.4 0.4 1.6 0.8 1.0
ROCKROLL a003466 0.0 0.0 0.4 0.0 0.0 0.6 0.2 0.4 0.2 0.0 0.2 0.8 0.2 0.6 0.4 0.4 0.2 0.2
ROCKROLL b000979 0.4 0.8 0.4 0.6 0.8 0.0 0.2 0.8 0.6 0.4 0.8 1.2 0.6 0.2 0.2 0.0 0.2 1.4
ROCKROLL b001750 0.2 0.2 0.2 0.4 0.4 0.4 0.4 0.4 0.6 0.6 0.2 0.4 0.8 0.2 0.4 0.0 0.6 0.2
ROCKROLL b005580 1.0 0.8 0.8 0.6 0.6 0.2 0.6 1.0 0.6 0.4 0.8 0.8 0.8 0.6 0.4 0.0 0.2 0.8
ROCKROLL b008059 1.2 1.2 1.2 1.0 1.0 0.8 0.4 0.6 1.0 0.6 1.2 1.6 1.0 0.6 1.0 0.2 0.4 1.0
ROCKROLL b008990 1.0 1.4 0.8 1.0 1.0 0.8 0.2 0.8 0.6 0.8 1.0 1.2 1.0 0.6 0.4 0.0 0.8 1.0
ROCKROLL b011113 1.0 1.0 1.0 1.0 1.0 0.4 0.8 1.0 0.8 0.6 0.8 1.0 1.0 0.6 1.0 0.4 1.0 1.0
ROCKROLL b016165 1.2 1.4 1.2 1.2 1.2 1.0 1.2 0.8 0.6 0.8 1.0 1.2 1.2 0.6 0.2 0.6 1.2 1.6
ROCKROLL b017586 1.2 1.2 1.2 1.0 1.0 1.4 0.8 1.0 1.0 1.0 1.4 1.2 1.2 0.0 0.6 0.4 0.8 0.6
ROCKROLL b019928 1.2 1.4 1.4 1.2 1.2 0.4 0.4 0.8 0.6 1.0 1.2 1.0 1.2 0.4 1.0 0.8 0.8 1.0
ROMANTIC d005834 1.2 1.2 1.4 1.2 1.2 0.6 0.0 0.2 0.4 0.2 1.4 1.4 1.2 0.0 0.0 0.6 0.8 1.0
ROMANTIC d009949 1.8 1.4 1.2 1.4 1.4 1.2 1.4 1.4 1.4 1.4 1.8 1.4 1.4 1.2 1.2 0.2 0.6 1.8
ROMANTIC d010217 1.0 1.0 1.0 1.0 1.0 0.6 1.0 1.0 0.8 0.8 1.2 1.0 1.2 1.0 1.0 0.4 0.8 1.2
ROMANTIC d012214 1.6 1.6 1.2 1.6 1.4 1.2 1.0 1.2 1.2 1.0 1.2 1.6 2.0 0.6 0.8 0.8 0.8 1.8
ROMANTIC d015538 2.0 2.0 2.0 1.8 2.0 1.0 1.2 1.4 1.2 1.6 2.0 2.0 1.8 1.4 0.6 0.8 1.8 2.0
ROMANTIC d016565 0.8 1.0 0.6 0.6 0.6 0.4 0.4 0.2 0.4 0.2 1.0 0.6 0.8 0.8 0.2 0.0 0.2 0.8
ROMANTIC d017427 2.0 2.0 2.0 2.0 2.0 1.4 1.2 1.0 0.8 0.8 2.0 2.0 2.0 1.2 0.8 0.8 0.8 1.6
ROMANTIC d017784 1.0 1.4 1.0 1.2 1.2 1.6 1.2 0.8 1.0 0.8 1.4 1.2 1.6 2.0 1.4 2.0 0.6 1.2
ROMANTIC d018721 1.2 1.4 1.2 0.8 0.8 0.4 1.6 1.2 1.2 0.8 1.6 1.2 1.4 0.6 0.6 0.4 0.2 1.4
ROMANTIC d019573 2.0 2.0 2.0 1.8 1.8 0.4 1.2 1.8 1.4 2.0 2.0 2.0 2.0 1.2 0.6 0.4 0.8 2.0

download these results as csv

Raw Scores

The raw data derived from the Evalutron 6000 human evaluations are located on the 2011:Audio Music Similarity and Retrieval Raw Data page.

Metadata and Distance Space Evaluation

The following reports provide evaluation statistics based on analysis of the distance space and metadata matches and include:

  • Neighbourhood clustering by artist, album and genre
  • Artist-filtered genre clustering
  • How often the triangular inequality holds
  • Statistics on 'hubs' (tracks similar to many tracks) and orphans (tracks that are not similar to any other tracks at N results).

Reports

STBD1 = Simone Sammartino, Cristina de la Bandera, Lorenzo J. Tardón, Isabel Barbancho
STBD2 = Simone Sammartino, Cristina de la Bandera, Lorenzo J. Tardón, Isabel Barbancho
STBD3 = Simone Sammartino, Cristina de la Bandera, Lorenzo J. Tardón, Isabel Barbancho
DM2 = Franz de Leon, Kirk Martinez
DM3 = Franz de Leon, Kirk Martinez
GKC1 = Aggelos Gkiokas, Vassilis Katsouros, George Carayannis
HKHLL1 = Byeong-jun Han, Kyogu Lee, Ziwon Hyung, Hyunwoo Kim, Sheayun Lee
ML1 = Brian McFee, Gert Lanckriet
ML2 = Brian McFee, Gert Lanckriet
ML3 = Brian McFee, Gert Lanckriet
PS1 = Dominik Schnitzer, Tim Pohle
SSKS3 = Klaus Seyerlehner, Markus Schedl, Peter Knees, Reinhard Sonnleitner
SSPK2 = Klaus Seyerlehner, Markus Schedl, Peter Knees, Tim Pohle
STBD1 = Simone Sammartino, Cristina de la Bandera, Lorenzo J. Tardón, Isabel Barbancho
STBD2 = Simone Sammartino, Cristina de la Bandera, Lorenzo J. Tardón, Isabel Barbancho
STBD3 = Simone Sammartino, Cristina de la Bandera, Lorenzo J. Tardón, Isabel Barbancho
YL1 = Yin-Tzu Lin, Wen-Huang Cheng, Ja-Ling Wu
ZYC2 = Dongying Zhang, Deshun Yang, Xiaoou Chen

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2012:Audio_Music_Similarity_and_Retrieval_Results&oldid=9071"