Difference between revisions of "2009:Audio Music Similarity and Retrieval Results"
From MIREX Wiki
IMIRSELBot (talk | contribs) m (Robot: Automated text replacement (-\[http://music-ir.org/mirex/2009/results/abs/ +[http://www.music-ir.org/mirex/abs/2009/)) |
(→Reports) |
||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 12: | Line 12: | ||
==== Team ID ==== | ==== Team ID ==== | ||
| − | '''ANO''' = [https://www.music-ir.org/mirex/ | + | '''ANO''' = [https://www.music-ir.org/mirex/abstracts/2009/ANO_train_simi.pdf Anonymous]<br /> |
| − | '''BF1''' = [https://www.music-ir.org/mirex/ | + | '''BF1''' = [https://www.music-ir.org/mirex/abstracts/2009/BF.pdf Benjamin Fields (chr12)]<br /> |
| − | '''BF2''' = [https://www.music-ir.org/mirex/ | + | '''BF2''' = [https://www.music-ir.org/mirex/abstracts/2009/BF.pdf Benjamin Fields (mfcc10)]<br /> |
| − | '''BSWH1''' = [https://www.music-ir.org/mirex/ | + | '''BSWH1''' = [https://www.music-ir.org/mirex/abstracts/2009/MIREX2009-sim-BSWH1-BSWH2.pdf Dmitry Bogdanov, Joan Serrà, Nicolas Wack, Perfecto Herrera (clas)]<br /> |
| − | '''BSWH2''' = [https://www.music-ir.org/mirex/ | + | '''BSWH2''' = [https://www.music-ir.org/mirex/abstracts/2009/MIREX2009-sim-BSWH1-BSWH2.pdf Dmitry Bogdanov, Joan Serrà, Nicolas Wack, Perfecto Herrera (hybrid)]<br /> |
| − | '''CL1''' = [https://www.music-ir.org/mirex/ | + | '''CL1''' = [https://www.music-ir.org/mirex/abstracts/2009/CL.pdf Chuan Cao, Ming Li]<br /> |
| − | '''CL2''' = [https://www.music-ir.org/mirex/ | + | '''CL2''' = [https://www.music-ir.org/mirex/abstracts/2009/CL.pdf Chuan Cao, Ming Li]<br /> |
| − | '''GT''' = [https://www.music-ir.org/mirex/ | + | '''GT''' = [https://www.music-ir.org/mirex/abstracts/2009/GTfinal.pdf George Tzanetakis]<br /> |
| − | '''LR''' = [https://www.music-ir.org/mirex/ | + | '''LR''' = [https://www.music-ir.org/mirex/abstracts/2009/LR.pdf Thomas Lidy, Andreas Rauber]<br /> |
| − | '''ME1''' = [https://www.music-ir.org/mirex/ | + | '''ME1''' = [https://www.music-ir.org/mirex/abstracts/2009/ME.pdf François Maillet, Douglas Eck (mlp)]<br /> |
| − | '''ME2''' = [https://www.music-ir.org/mirex/ | + | '''ME2''' = [https://www.music-ir.org/mirex/abstracts/2009/ME.pdf François Maillet, Douglas Eck (sda)]<br /> |
| − | '''PS1''' = [https://www.music-ir.org/mirex/ | + | '''PS1''' = [https://www.music-ir.org/mirex/abstracts/2009/PS.pdf Tim Pohle, Dominik Schnitzer (2007)]<br /> |
| − | '''PS2''' = [https://www.music-ir.org/mirex/ | + | '''PS2''' = [https://www.music-ir.org/mirex/abstracts/2009/PS.pdf Tim Pohle, Dominik Schnitzer (2009)]<br /> |
| − | '''SH1''' = [https://www.music-ir.org/mirex/ | + | '''SH1''' = [https://www.music-ir.org/mirex/abstracts/2009/SH.pdf Stephan Hübler]<br /> |
| − | '''SH2''' = [https://www.music-ir.org/mirex/ | + | '''SH2''' = [https://www.music-ir.org/mirex/abstracts/2009/SH.pdf Stephan Hübler]<br /> |
====Broad Categories==== | ====Broad Categories==== | ||
| Line 49: | Line 49: | ||
<csv p=3>2009/ams/evalutron/evalutron.fine.friedman.tukeyKramerHSD.csv</csv> | <csv p=3>2009/ams/evalutron/evalutron.fine.friedman.tukeyKramerHSD.csv</csv> | ||
| − | https://music-ir.org/mirex/2009 | + | https://music-ir.org/mirex/results/2009/ams/evalutron/small.evalutron.fine.friedman.tukeyKramerHSD.png |
====Friedman's Test (BROAD Scores)==== | ====Friedman's Test (BROAD Scores)==== | ||
| Line 57: | Line 57: | ||
<csv p=3>2009/ams/evalutron/evalutron.cat.friedman.tukeyKramerHSD.csv</csv> | <csv p=3>2009/ams/evalutron/evalutron.cat.friedman.tukeyKramerHSD.csv</csv> | ||
| − | https://music-ir.org/mirex/2009 | + | https://music-ir.org/mirex/results/2009/ams/evalutron/small.evalutron.cat.friedman.tukeyKramerHSD.png |
| Line 85: | Line 85: | ||
=== Reports === | === Reports === | ||
| − | '''ANO''' = [https://music-ir.org/mirex/2009 | + | '''ANO''' = [https://music-ir.org/mirex/results/2009/ams/statistics/ANO/report.txt Anonymous]<br /> |
| − | '''BF1''' = [https://music-ir.org/mirex/2009 | + | '''BF1''' = [https://music-ir.org/mirex/results/2009/ams/statistics/BF1/report.txt Benjamin Fields (chr12)]<br /> |
| − | '''BF2''' = [https://music-ir.org/mirex/2009 | + | '''BF2''' = [https://music-ir.org/mirex/results/2009/ams/statistics/BF2/report.txt Benjamin Fields (mfcc10)]<br /> |
| − | '''BSWH1''' = [https://music-ir.org/mirex/2009 | + | '''BSWH1''' = [https://music-ir.org/mirex/results/2009/ams/statistics/BSWH1/report.txt Dmitry Bogdanov, Joan Serrà, Nicolas Wack, and Perfecto Herrera (clas)]<br /> |
| − | '''BSWH2''' = [https://music-ir.org/mirex/2009 | + | '''BSWH2''' = [https://music-ir.org/mirex/results/2009/ams/statistics/BSWH2/report.txt Dmitry Bogdanov, Joan Serrà, Nicolas Wack, and Perfecto Herrera (hybrid)]<br /> |
| − | '''CL1''' = [https://music-ir.org/mirex/2009 | + | '''CL1''' = [https://music-ir.org/mirex/results/2009/ams/statistics/CL1/report.txt Chuan Cao, Ming Li]<br /> |
| − | '''CL2''' = [https://music-ir.org/mirex/2009 | + | '''CL2''' = [https://music-ir.org/mirex/results/2009/ams/statistics/CL2/report.txt Chuan Cao, Ming Li]<br /> |
| − | '''GT''' = [https://music-ir.org/mirex/2009 | + | '''GT''' = [https://music-ir.org/mirex/results/2009/ams/statistics/GT/report.txt George Tzanetakis]<br /> |
| − | '''LR''' = [https://music-ir.org/mirex/2009 | + | '''LR''' = [https://music-ir.org/mirex/results/2009/ams/statistics/LR/report.txt Thomas Lidy, Andreas Rauber]]<br /> |
| − | '''ME1''' = [https://music-ir.org/mirex/2009 | + | '''ME1''' = [https://music-ir.org/mirex/results/2009/ams/statistics/ME1/report.txt François Maillet, Douglas Eck (mlp)]<br /> |
| − | '''ME2''' = [https://music-ir.org/mirex/2009 | + | '''ME2''' = [https://music-ir.org/mirex/results/2009/ams/statistics/ME2/report.txt François Maillet, Douglas Eck (sda)]<br /> |
| − | '''PS1''' = [https://music-ir.org/mirex/2009 | + | '''PS1''' = [https://music-ir.org/mirex/results/2009/ams/statistics/PS1/report.txt Tim Pohle, Dominik Schnitzer (2007)]<br /> |
| − | '''PS2''' = [https://music-ir.org/mirex/2009 | + | '''PS2''' = [https://music-ir.org/mirex/results/2009/ams/statistics/PS2/report.txt Tim Pohle, Dominik Schnitzer (2009)]<br /> |
| − | '''SH1''' = [https://music-ir.org/mirex/2009 | + | '''SH1''' = [https://music-ir.org/mirex/results/2009/ams/statistics/SH1/report.txt Stephan Hübler]<br /> |
| − | '''SH2''' = [https://music-ir.org/mirex/2009 | + | '''SH2''' = [https://music-ir.org/mirex/results/2009/ams/statistics/SH2/report.txt Stephan Hübler]<br /> |
== Run Times == | == Run Times == | ||
<csv>2009/ams/audiosim.runtime.csv</csv> | <csv>2009/ams/audiosim.runtime.csv</csv> | ||
Latest revision as of 15:31, 23 July 2010
Introduction
These are the results for the 2009 running of the Audio Music Similarity and Retrieval task set. For background information about this task set please refer to the Audio Music Similarity and Retrieval page.
Each system was given 7000 songs chosen from IMIRSEL's "uspop", "uscrap" and "american" "classical" and "sundry" collections. Each system then returned a 7000x7000 distance matrix. 100 songs were randomly selected from the 10 genre groups (10 per genre) as queries and the first 5 most highly ranked songs out of the 7000 were extracted for each query (after filtering out the query itself, returned results from the same artist were also omitted). Then, for each query, the returned results (candidates) from all participants were grouped and were evaluated by human graders using the Evalutron 6000 grading system. Each individual query/candidate set was evaluated by a single grader. For each query/candidate pair, graders provided two scores. Graders were asked to provide 1 categorical BROAD score with 3 categories: NS,SS,VS as explained below, and one FINE score (in the range from 0 to 10). A description and analysis is provided below.
The systems read in 30 second audio clips as their raw data. The same 30 second clips were used in the grading stage.
General Legend
Team ID
ANO = Anonymous
BF1 = Benjamin Fields (chr12)
BF2 = Benjamin Fields (mfcc10)
BSWH1 = Dmitry Bogdanov, Joan Serrà, Nicolas Wack, Perfecto Herrera (clas)
BSWH2 = Dmitry Bogdanov, Joan Serrà, Nicolas Wack, Perfecto Herrera (hybrid)
CL1 = Chuan Cao, Ming Li
CL2 = Chuan Cao, Ming Li
GT = George Tzanetakis
LR = Thomas Lidy, Andreas Rauber
ME1 = François Maillet, Douglas Eck (mlp)
ME2 = François Maillet, Douglas Eck (sda)
PS1 = Tim Pohle, Dominik Schnitzer (2007)
PS2 = Tim Pohle, Dominik Schnitzer (2009)
SH1 = Stephan Hübler
SH2 = Stephan Hübler
Broad Categories
NS = Not Similar
SS = Somewhat Similar
VS = Very Similar
Understanding Summary Measures
Fine = Has a range from 0 (failure) to 10 (perfection).
Broad = Has a range from 0 (failure) to 2 (perfection) as each query/candidate pair is scored with either NS=0, SS=1 or VS=2.
Human Evaluation
Overall Summary Results
| Measure | ANO | BF1 | BF2 | BSWH1 | BSWH2 | CL1 | CL2 | GT | LR | ME1 | ME2 | PS1 | PS2 | SH1 | SH2 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Average FINE Score | 5.391 | 2.401 | 2.587 | 5.137 | 5.734 | 2.525 | 5.392 | 5.343 | 5.470 | 2.331 | 2.585 | 5.751 | 6.458 | 5.042 | 4.932 |
| Average BROAD Score | 1.126 | 0.416 | 0.410 | 1.094 | 1.232 | 0.476 | 1.164 | 1.126 | 1.148 | 0.356 | 0.418 | 1.262 | 1.448 | 1.012 | 1.040 |
Friedman's Tests
Friedman's Test (FINE Scores)
The Friedman test was run in MATLAB against the Fine summary data over the 100 queries.
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
| TeamID | TeamID | Lowerbound | Mean | Upperbound | Significance |
|---|---|---|---|---|---|
| PS2 | PS1 | -0.298 | 1.845 | 3.988 | FALSE |
| PS2 | BSWH2 | -0.278 | 1.865 | 4.008 | FALSE |
| PS2 | LR | 0.502 | 2.645 | 4.788 | TRUE |
| PS2 | CL2 | 0.917 | 3.060 | 5.203 | TRUE |
| PS2 | ANO | 0.767 | 2.910 | 5.053 | TRUE |
| PS2 | GT | 0.817 | 2.960 | 5.103 | TRUE |
| PS2 | BSWH1 | 1.642 | 3.785 | 5.928 | TRUE |
| PS2 | SH1 | 1.617 | 3.760 | 5.903 | TRUE |
| PS2 | SH2 | 1.837 | 3.980 | 6.123 | TRUE |
| PS2 | BF2 | 6.507 | 8.650 | 10.793 | TRUE |
| PS2 | ME2 | 6.557 | 8.700 | 10.843 | TRUE |
| PS2 | CL1 | 6.627 | 8.770 | 10.913 | TRUE |
| PS2 | BF1 | 6.857 | 9.000 | 11.143 | TRUE |
| PS2 | ME1 | 6.952 | 9.095 | 11.238 | TRUE |
| PS1 | BSWH2 | -2.123 | 0.020 | 2.163 | FALSE |
| PS1 | LR | -1.343 | 0.800 | 2.943 | FALSE |
| PS1 | CL2 | -0.928 | 1.215 | 3.358 | FALSE |
| PS1 | ANO | -1.078 | 1.065 | 3.208 | FALSE |
| PS1 | GT | -1.028 | 1.115 | 3.258 | FALSE |
| PS1 | BSWH1 | -0.203 | 1.940 | 4.083 | FALSE |
| PS1 | SH1 | -0.228 | 1.915 | 4.058 | FALSE |
| PS1 | SH2 | -0.008 | 2.135 | 4.278 | FALSE |
| PS1 | BF2 | 4.662 | 6.805 | 8.948 | TRUE |
| PS1 | ME2 | 4.712 | 6.855 | 8.998 | TRUE |
| PS1 | CL1 | 4.782 | 6.925 | 9.068 | TRUE |
| PS1 | BF1 | 5.012 | 7.155 | 9.298 | TRUE |
| PS1 | ME1 | 5.107 | 7.250 | 9.393 | TRUE |
| BSWH2 | LR | -1.363 | 0.780 | 2.923 | FALSE |
| BSWH2 | CL2 | -0.948 | 1.195 | 3.338 | FALSE |
| BSWH2 | ANO | -1.098 | 1.045 | 3.188 | FALSE |
| BSWH2 | GT | -1.048 | 1.095 | 3.238 | FALSE |
| BSWH2 | BSWH1 | -0.223 | 1.920 | 4.063 | FALSE |
| BSWH2 | SH1 | -0.248 | 1.895 | 4.038 | FALSE |
| BSWH2 | SH2 | -0.028 | 2.115 | 4.258 | FALSE |
| BSWH2 | BF2 | 4.642 | 6.785 | 8.928 | TRUE |
| BSWH2 | ME2 | 4.692 | 6.835 | 8.978 | TRUE |
| BSWH2 | CL1 | 4.762 | 6.905 | 9.048 | TRUE |
| BSWH2 | BF1 | 4.992 | 7.135 | 9.278 | TRUE |
| BSWH2 | ME1 | 5.087 | 7.230 | 9.373 | TRUE |
| LR | CL2 | -1.728 | 0.415 | 2.558 | FALSE |
| LR | ANO | -1.878 | 0.265 | 2.408 | FALSE |
| LR | GT | -1.828 | 0.315 | 2.458 | FALSE |
| LR | BSWH1 | -1.003 | 1.140 | 3.283 | FALSE |
| LR | SH1 | -1.028 | 1.115 | 3.258 | FALSE |
| LR | SH2 | -0.808 | 1.335 | 3.478 | FALSE |
| LR | BF2 | 3.862 | 6.005 | 8.148 | TRUE |
| LR | ME2 | 3.912 | 6.055 | 8.198 | TRUE |
| LR | CL1 | 3.982 | 6.125 | 8.268 | TRUE |
| LR | BF1 | 4.212 | 6.355 | 8.498 | TRUE |
| LR | ME1 | 4.307 | 6.450 | 8.593 | TRUE |
| CL2 | ANO | -2.293 | -0.150 | 1.993 | FALSE |
| CL2 | GT | -2.243 | -0.100 | 2.043 | FALSE |
| CL2 | BSWH1 | -1.418 | 0.725 | 2.868 | FALSE |
| CL2 | SH1 | -1.443 | 0.700 | 2.843 | FALSE |
| CL2 | SH2 | -1.223 | 0.920 | 3.063 | FALSE |
| CL2 | BF2 | 3.447 | 5.590 | 7.733 | TRUE |
| CL2 | ME2 | 3.497 | 5.640 | 7.783 | TRUE |
| CL2 | CL1 | 3.567 | 5.710 | 7.853 | TRUE |
| CL2 | BF1 | 3.797 | 5.940 | 8.083 | TRUE |
| CL2 | ME1 | 3.892 | 6.035 | 8.178 | TRUE |
| ANO | GT | -2.093 | 0.050 | 2.193 | FALSE |
| ANO | BSWH1 | -1.268 | 0.875 | 3.018 | FALSE |
| ANO | SH1 | -1.293 | 0.850 | 2.993 | FALSE |
| ANO | SH2 | -1.073 | 1.070 | 3.213 | FALSE |
| ANO | BF2 | 3.597 | 5.740 | 7.883 | TRUE |
| ANO | ME2 | 3.647 | 5.790 | 7.933 | TRUE |
| ANO | CL1 | 3.717 | 5.860 | 8.003 | TRUE |
| ANO | BF1 | 3.947 | 6.090 | 8.233 | TRUE |
| ANO | ME1 | 4.042 | 6.185 | 8.328 | TRUE |
| GT | BSWH1 | -1.318 | 0.825 | 2.968 | FALSE |
| GT | SH1 | -1.343 | 0.800 | 2.943 | FALSE |
| GT | SH2 | -1.123 | 1.020 | 3.163 | FALSE |
| GT | BF2 | 3.547 | 5.690 | 7.833 | TRUE |
| GT | ME2 | 3.597 | 5.740 | 7.883 | TRUE |
| GT | CL1 | 3.667 | 5.810 | 7.953 | TRUE |
| GT | BF1 | 3.897 | 6.040 | 8.183 | TRUE |
| GT | ME1 | 3.992 | 6.135 | 8.278 | TRUE |
| BSWH1 | SH1 | -2.168 | -0.025 | 2.118 | FALSE |
| BSWH1 | SH2 | -1.948 | 0.195 | 2.338 | FALSE |
| BSWH1 | BF2 | 2.722 | 4.865 | 7.008 | TRUE |
| BSWH1 | ME2 | 2.772 | 4.915 | 7.058 | TRUE |
| BSWH1 | CL1 | 2.842 | 4.985 | 7.128 | TRUE |
| BSWH1 | BF1 | 3.072 | 5.215 | 7.358 | TRUE |
| BSWH1 | ME1 | 3.167 | 5.310 | 7.453 | TRUE |
| SH1 | SH2 | -1.923 | 0.220 | 2.363 | FALSE |
| SH1 | BF2 | 2.747 | 4.890 | 7.033 | TRUE |
| SH1 | ME2 | 2.797 | 4.940 | 7.083 | TRUE |
| SH1 | CL1 | 2.867 | 5.010 | 7.153 | TRUE |
| SH1 | BF1 | 3.097 | 5.240 | 7.383 | TRUE |
| SH1 | ME1 | 3.192 | 5.335 | 7.478 | TRUE |
| SH2 | BF2 | 2.527 | 4.670 | 6.813 | TRUE |
| SH2 | ME2 | 2.577 | 4.720 | 6.863 | TRUE |
| SH2 | CL1 | 2.647 | 4.790 | 6.933 | TRUE |
| SH2 | BF1 | 2.877 | 5.020 | 7.163 | TRUE |
| SH2 | ME1 | 2.972 | 5.115 | 7.258 | TRUE |
| BF2 | ME2 | -2.093 | 0.050 | 2.193 | FALSE |
| BF2 | CL1 | -2.023 | 0.120 | 2.263 | FALSE |
| BF2 | BF1 | -1.793 | 0.350 | 2.493 | FALSE |
| BF2 | ME1 | -1.698 | 0.445 | 2.588 | FALSE |
| ME2 | CL1 | -2.073 | 0.070 | 2.213 | FALSE |
| ME2 | BF1 | -1.843 | 0.300 | 2.443 | FALSE |
| ME2 | ME1 | -1.748 | 0.395 | 2.538 | FALSE |
| CL1 | BF1 | -1.913 | 0.230 | 2.373 | FALSE |
| CL1 | ME1 | -1.818 | 0.325 | 2.468 | FALSE |
| BF1 | ME1 | -2.048 | 0.095 | 2.238 | FALSE |
Friedman's Test (BROAD Scores)
The Friedman test was run in MATLAB against the BROAD summary data over the 100 queries.
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
| TeamID | TeamID | Lowerbound | Mean | Upperbound | Significance |
|---|---|---|---|---|---|
| PS2 | PS1 | -0.352 | 1.730 | 3.812 | FALSE |
| PS2 | BSWH2 | -0.052 | 2.030 | 4.112 | FALSE |
| PS2 | CL2 | 0.657 | 2.740 | 4.822 | TRUE |
| PS2 | LR | 0.682 | 2.765 | 4.848 | TRUE |
| PS2 | ANO | 1.157 | 3.240 | 5.322 | TRUE |
| PS2 | GT | 0.802 | 2.885 | 4.968 | TRUE |
| PS2 | BSWH1 | 1.387 | 3.470 | 5.553 | TRUE |
| PS2 | SH2 | 1.742 | 3.825 | 5.907 | TRUE |
| PS2 | SH1 | 2.252 | 4.335 | 6.418 | TRUE |
| PS2 | CL1 | 6.277 | 8.360 | 10.443 | TRUE |
| PS2 | ME2 | 6.478 | 8.560 | 10.643 | TRUE |
| PS2 | BF1 | 6.402 | 8.485 | 10.568 | TRUE |
| PS2 | BF2 | 6.463 | 8.545 | 10.627 | TRUE |
| PS2 | ME1 | 6.772 | 8.855 | 10.938 | TRUE |
| PS1 | BSWH2 | -1.782 | 0.300 | 2.382 | FALSE |
| PS1 | CL2 | -1.073 | 1.010 | 3.092 | FALSE |
| PS1 | LR | -1.048 | 1.035 | 3.118 | FALSE |
| PS1 | ANO | -0.573 | 1.510 | 3.592 | FALSE |
| PS1 | GT | -0.927 | 1.155 | 3.237 | FALSE |
| PS1 | BSWH1 | -0.343 | 1.740 | 3.822 | FALSE |
| PS1 | SH2 | 0.013 | 2.095 | 4.178 | TRUE |
| PS1 | SH1 | 0.522 | 2.605 | 4.688 | TRUE |
| PS1 | CL1 | 4.548 | 6.630 | 8.713 | TRUE |
| PS1 | ME2 | 4.747 | 6.830 | 8.912 | TRUE |
| PS1 | BF1 | 4.673 | 6.755 | 8.838 | TRUE |
| PS1 | BF2 | 4.732 | 6.815 | 8.898 | TRUE |
| PS1 | ME1 | 5.043 | 7.125 | 9.207 | TRUE |
| BSWH2 | CL2 | -1.373 | 0.710 | 2.792 | FALSE |
| BSWH2 | LR | -1.347 | 0.735 | 2.817 | FALSE |
| BSWH2 | ANO | -0.873 | 1.210 | 3.292 | FALSE |
| BSWH2 | GT | -1.228 | 0.855 | 2.938 | FALSE |
| BSWH2 | BSWH1 | -0.642 | 1.440 | 3.522 | FALSE |
| BSWH2 | SH2 | -0.287 | 1.795 | 3.877 | FALSE |
| BSWH2 | SH1 | 0.223 | 2.305 | 4.388 | TRUE |
| BSWH2 | CL1 | 4.247 | 6.330 | 8.412 | TRUE |
| BSWH2 | ME2 | 4.447 | 6.530 | 8.613 | TRUE |
| BSWH2 | BF1 | 4.372 | 6.455 | 8.537 | TRUE |
| BSWH2 | BF2 | 4.433 | 6.515 | 8.598 | TRUE |
| BSWH2 | ME1 | 4.742 | 6.825 | 8.908 | TRUE |
| CL2 | LR | -2.058 | 0.025 | 2.107 | FALSE |
| CL2 | ANO | -1.583 | 0.500 | 2.583 | FALSE |
| CL2 | GT | -1.938 | 0.145 | 2.228 | FALSE |
| CL2 | BSWH1 | -1.353 | 0.730 | 2.812 | FALSE |
| CL2 | SH2 | -0.998 | 1.085 | 3.167 | FALSE |
| CL2 | SH1 | -0.487 | 1.595 | 3.678 | FALSE |
| CL2 | CL1 | 3.538 | 5.620 | 7.702 | TRUE |
| CL2 | ME2 | 3.737 | 5.820 | 7.902 | TRUE |
| CL2 | BF1 | 3.663 | 5.745 | 7.827 | TRUE |
| CL2 | BF2 | 3.723 | 5.805 | 7.888 | TRUE |
| CL2 | ME1 | 4.032 | 6.115 | 8.197 | TRUE |
| LR | ANO | -1.607 | 0.475 | 2.558 | FALSE |
| LR | GT | -1.962 | 0.120 | 2.203 | FALSE |
| LR | BSWH1 | -1.377 | 0.705 | 2.788 | FALSE |
| LR | SH2 | -1.022 | 1.060 | 3.143 | FALSE |
| LR | SH1 | -0.512 | 1.570 | 3.652 | FALSE |
| LR | CL1 | 3.513 | 5.595 | 7.678 | TRUE |
| LR | ME2 | 3.712 | 5.795 | 7.878 | TRUE |
| LR | BF1 | 3.638 | 5.720 | 7.803 | TRUE |
| LR | BF2 | 3.697 | 5.780 | 7.862 | TRUE |
| LR | ME1 | 4.008 | 6.090 | 8.172 | TRUE |
| ANO | GT | -2.438 | -0.355 | 1.728 | FALSE |
| ANO | BSWH1 | -1.853 | 0.230 | 2.312 | FALSE |
| ANO | SH2 | -1.498 | 0.585 | 2.667 | FALSE |
| ANO | SH1 | -0.988 | 1.095 | 3.178 | FALSE |
| ANO | CL1 | 3.038 | 5.120 | 7.202 | TRUE |
| ANO | ME2 | 3.237 | 5.320 | 7.402 | TRUE |
| ANO | BF1 | 3.163 | 5.245 | 7.327 | TRUE |
| ANO | BF2 | 3.223 | 5.305 | 7.388 | TRUE |
| ANO | ME1 | 3.533 | 5.615 | 7.697 | TRUE |
| GT | BSWH1 | -1.498 | 0.585 | 2.667 | FALSE |
| GT | SH2 | -1.143 | 0.940 | 3.022 | FALSE |
| GT | SH1 | -0.632 | 1.450 | 3.533 | FALSE |
| GT | CL1 | 3.393 | 5.475 | 7.558 | TRUE |
| GT | ME2 | 3.592 | 5.675 | 7.758 | TRUE |
| GT | BF1 | 3.518 | 5.600 | 7.683 | TRUE |
| GT | BF2 | 3.578 | 5.660 | 7.742 | TRUE |
| GT | ME1 | 3.888 | 5.970 | 8.053 | TRUE |
| BSWH1 | SH2 | -1.728 | 0.355 | 2.438 | FALSE |
| BSWH1 | SH1 | -1.218 | 0.865 | 2.947 | FALSE |
| BSWH1 | CL1 | 2.808 | 4.890 | 6.973 | TRUE |
| BSWH1 | ME2 | 3.007 | 5.090 | 7.173 | TRUE |
| BSWH1 | BF1 | 2.933 | 5.015 | 7.098 | TRUE |
| BSWH1 | BF2 | 2.993 | 5.075 | 7.157 | TRUE |
| BSWH1 | ME1 | 3.303 | 5.385 | 7.468 | TRUE |
| SH2 | SH1 | -1.573 | 0.510 | 2.592 | FALSE |
| SH2 | CL1 | 2.453 | 4.535 | 6.617 | TRUE |
| SH2 | ME2 | 2.652 | 4.735 | 6.817 | TRUE |
| SH2 | BF1 | 2.578 | 4.660 | 6.742 | TRUE |
| SH2 | BF2 | 2.638 | 4.720 | 6.803 | TRUE |
| SH2 | ME1 | 2.947 | 5.030 | 7.112 | TRUE |
| SH1 | CL1 | 1.942 | 4.025 | 6.107 | TRUE |
| SH1 | ME2 | 2.143 | 4.225 | 6.308 | TRUE |
| SH1 | BF1 | 2.067 | 4.150 | 6.232 | TRUE |
| SH1 | BF2 | 2.127 | 4.210 | 6.293 | TRUE |
| SH1 | ME1 | 2.438 | 4.520 | 6.603 | TRUE |
| CL1 | ME2 | -1.883 | 0.200 | 2.283 | FALSE |
| CL1 | BF1 | -1.958 | 0.125 | 2.208 | FALSE |
| CL1 | BF2 | -1.897 | 0.185 | 2.268 | FALSE |
| CL1 | ME1 | -1.587 | 0.495 | 2.578 | FALSE |
| ME2 | BF1 | -2.158 | -0.075 | 2.007 | FALSE |
| ME2 | BF2 | -2.098 | -0.015 | 2.067 | FALSE |
| ME2 | ME1 | -1.788 | 0.295 | 2.377 | FALSE |
| BF1 | BF2 | -2.022 | 0.060 | 2.143 | FALSE |
| BF1 | ME1 | -1.712 | 0.370 | 2.453 | FALSE |
| BF2 | ME1 | -1.772 | 0.310 | 2.393 | FALSE |
Summary Results by Query
FINE Scores
These are the mean FINE scores per query assigned by Evalutron graders. The FINE scores for the 5 candidates returned per algorithm, per query, have been averaged. Values are bounded between 0.0 and 10.0. A perfect score would be 10. Genre labels have been included for reference.
| Genre | Query | ANO | BF1 | BF2 | BSWH1 | BSWH2 | CL1 | CL2 | GT | LR | ME1 | ME2 | PS1 | PS2 | SH1 | SH2 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BAROQUE | d005166 | 3.060 | 2.120 | 1.900 | 2.520 | 3.700 | 3.280 | 5.560 | 3.560 | 2.620 | 0.540 | 1.860 | 3.040 | 3.700 | 3.960 | 2.040 |
| BAROQUE | d007244 | 5.260 | 0.540 | 2.620 | 6.060 | 5.540 | 4.400 | 5.940 | 5.900 | 6.140 | 3.500 | 1.840 | 4.880 | 7.100 | 5.180 | 6.360 |
| BAROQUE | d004490 | 7.880 | 3.940 | 6.680 | 3.400 | 6.060 | 5.340 | 8.520 | 7.500 | 8.300 | 5.160 | 3.440 | 7.800 | 9.200 | 7.080 | 8.280 |
| BAROQUE | d009737 | 7.620 | 5.180 | 0.420 | 4.540 | 8.940 | 5.860 | 6.080 | 2.480 | 8.180 | 1.320 | 1.060 | 8.820 | 8.840 | 3.440 | 2.780 |
| BAROQUE | d009837 | 4.140 | 1.520 | 0.360 | 1.700 | 1.640 | 2.860 | 1.000 | 3.100 | 3.860 | 0.700 | 0.720 | 3.960 | 4.680 | 2.320 | 3.520 |
| BAROQUE | d008174 | 5.000 | 3.880 | 2.260 | 7.680 | 7.600 | 7.140 | 7.460 | 7.480 | 7.020 | 3.320 | 1.860 | 7.700 | 7.680 | 5.080 | 7.580 |
| BAROQUE | d011054 | 7.320 | 4.160 | 0.000 | 4.360 | 6.140 | 5.500 | 5.280 | 5.460 | 5.980 | 0.940 | 1.000 | 5.980 | 7.160 | 5.780 | 5.560 |
| BAROQUE | d018083 | 6.940 | 1.380 | 2.540 | 5.600 | 3.040 | 2.540 | 5.940 | 3.400 | 6.240 | 0.460 | 0.260 | 3.660 | 3.580 | 3.440 | 4.020 |
| BAROQUE | d017335 | 4.680 | 0.280 | 0.960 | 1.940 | 4.640 | 2.340 | 3.440 | 5.780 | 3.380 | 2.540 | 0.000 | 3.560 | 4.240 | 3.880 | 1.980 |
| BAROQUE | d019774 | 5.500 | 3.600 | 4.660 | 4.060 | 4.000 | 3.860 | 4.400 | 5.560 | 4.060 | 3.440 | 5.160 | 4.500 | 6.300 | 4.960 | 4.820 |
| BLUES | e002545 | 5.760 | 6.260 | 1.320 | 5.600 | 7.140 | 0.280 | 6.760 | 6.140 | 5.160 | 1.600 | 4.180 | 7.340 | 8.060 | 4.760 | 5.200 |
| BLUES | e000111 | 3.200 | 5.900 | 4.300 | 3.800 | 3.200 | 1.980 | 7.400 | 5.500 | 3.000 | 2.900 | 7.000 | 6.000 | 6.660 | 7.300 | 6.800 |
| BLUES | e001417 | 6.700 | 3.320 | 2.920 | 4.760 | 4.980 | 7.480 | 8.800 | 7.660 | 7.000 | 6.480 | 4.940 | 7.160 | 6.980 | 5.020 | 7.160 |
| BLUES | e004211 | 2.880 | 2.840 | 2.680 | 2.720 | 5.880 | 6.840 | 6.520 | 4.280 | 5.380 | 0.180 | 2.020 | 7.840 | 8.460 | 6.640 | 0.120 |
| BLUES | e014267 | 5.400 | 2.200 | 0.960 | 7.040 | 4.700 | 0.000 | 4.880 | 5.400 | 2.780 | 1.380 | 3.220 | 4.460 | 6.640 | 4.440 | 2.280 |
| BLUES | e013973 | 8.480 | 0.700 | 4.140 | 6.680 | 8.100 | 0.480 | 9.060 | 8.560 | 8.400 | 4.300 | 3.560 | 8.520 | 8.520 | 8.660 | 8.140 |
| BLUES | e014486 | 3.800 | 3.080 | 2.280 | 4.280 | 5.180 | 0.160 | 1.760 | 5.380 | 3.160 | 3.420 | 1.300 | 5.920 | 4.360 | 4.660 | 5.000 |
| BLUES | e010067 | 4.160 | 2.180 | 2.520 | 5.640 | 7.320 | 4.620 | 6.620 | 5.480 | 5.100 | 7.020 | 2.960 | 4.680 | 7.960 | 3.880 | 4.460 |
| BLUES | e012895 | 7.760 | 3.360 | 1.580 | 5.880 | 7.560 | 6.720 | 7.740 | 6.460 | 7.840 | 2.280 | 1.740 | 8.360 | 8.540 | 7.700 | 6.060 |
| BLUES | e015354 | 5.400 | 2.700 | 0.000 | 4.000 | 5.600 | 0.000 | 6.400 | 4.800 | 4.800 | 0.800 | 2.200 | 1.200 | 6.900 | 5.600 | 6.800 |
| CLASSICAL | d000239 | 7.580 | 5.080 | 0.720 | 6.400 | 7.280 | 7.640 | 6.160 | 9.100 | 8.020 | 0.580 | 3.620 | 8.860 | 8.760 | 5.360 | 6.500 |
| CLASSICAL | d000762 | 6.100 | 2.040 | 2.520 | 7.200 | 6.840 | 5.260 | 6.660 | 6.660 | 6.940 | 0.800 | 0.720 | 8.300 | 8.520 | 5.400 | 5.800 |
| CLASSICAL | d002538 | 7.200 | 0.200 | 3.800 | 6.600 | 7.000 | 4.200 | 6.400 | 6.800 | 7.000 | 2.400 | 3.800 | 6.800 | 7.200 | 6.600 | 6.800 |
| CLASSICAL | d001502 | 8.700 | 4.300 | 7.560 | 8.600 | 8.900 | 8.400 | 8.820 | 8.840 | 9.100 | 6.740 | 5.800 | 8.920 | 8.800 | 9.100 | 8.660 |
| CLASSICAL | d012972 | 7.960 | 3.960 | 2.080 | 8.100 | 7.860 | 6.860 | 7.640 | 8.820 | 7.200 | 3.860 | 4.640 | 8.420 | 7.940 | 7.660 | 7.580 |
| CLASSICAL | d010713 | 5.000 | 3.820 | 2.500 | 5.100 | 5.200 | 4.200 | 1.740 | 3.600 | 3.100 | 2.400 | 4.100 | 2.800 | 3.300 | 3.200 | 3.100 |
| CLASSICAL | d012985 | 8.680 | 4.200 | 1.920 | 3.120 | 9.080 | 5.900 | 8.040 | 6.460 | 8.740 | 0.420 | 5.520 | 8.760 | 8.920 | 7.720 | 8.240 |
| CLASSICAL | d019802 | 6.660 | 2.060 | 1.500 | 5.580 | 7.380 | 4.680 | 5.440 | 4.680 | 3.100 | 2.300 | 3.540 | 5.700 | 6.220 | 6.420 | 6.540 |
| CLASSICAL | d019790 | 6.600 | 5.200 | 6.000 | 6.600 | 7.200 | 4.800 | 6.200 | 5.400 | 6.200 | 1.800 | 3.400 | 7.000 | 6.600 | 6.600 | 6.800 |
| CLASSICAL | d019783 | 7.800 | 2.400 | 4.200 | 6.800 | 7.400 | 7.000 | 7.800 | 6.400 | 7.600 | 3.600 | 2.200 | 6.100 | 8.400 | 8.200 | 7.800 |
| COUNTRY | e002580 | 3.880 | 1.660 | 2.300 | 3.740 | 3.340 | 0.000 | 2.440 | 4.800 | 2.900 | 3.080 | 1.920 | 3.760 | 3.820 | 2.640 | 2.500 |
| COUNTRY | e003090 | 3.200 | 0.800 | 2.800 | 2.200 | 4.000 | 0.000 | 2.200 | 3.800 | 4.400 | 1.600 | 0.800 | 6.200 | 7.200 | 3.000 | 2.500 |
| COUNTRY | e001591 | 2.000 | 4.760 | 3.780 | 3.380 | 6.340 | 1.700 | 4.880 | 5.140 | 2.740 | 2.640 | 3.060 | 7.200 | 6.680 | 5.600 | 5.500 |
| COUNTRY | e006811 | 4.820 | 2.480 | 3.860 | 5.400 | 5.760 | 0.860 | 4.220 | 3.580 | 5.040 | 1.460 | 2.800 | 4.900 | 8.360 | 6.040 | 3.460 |
| COUNTRY | e003155 | 0.000 | 0.060 | 1.200 | 0.200 | 2.800 | 0.000 | 3.340 | 1.720 | 2.100 | 0.200 | 0.500 | 3.700 | 5.320 | 5.220 | 2.680 |
| COUNTRY | e003544 | 6.200 | 0.880 | 2.660 | 5.200 | 3.720 | 0.000 | 4.040 | 4.600 | 4.500 | 1.100 | 2.900 | 5.400 | 5.240 | 5.720 | 6.280 |
| COUNTRY | e014385 | 4.800 | 0.000 | 0.600 | 2.600 | 3.800 | 0.000 | 2.000 | 2.900 | 1.700 | 0.900 | 1.700 | 5.300 | 5.000 | 1.800 | 1.820 |
| COUNTRY | e011855 | 6.860 | 0.740 | 1.160 | 4.280 | 6.340 | 0.440 | 4.960 | 6.200 | 6.560 | 1.500 | 0.820 | 7.380 | 5.040 | 5.480 | 5.120 |
| COUNTRY | e015843 | 5.540 | 1.940 | 2.140 | 3.620 | 2.880 | 0.000 | 4.400 | 4.980 | 4.400 | 2.180 | 4.780 | 5.500 | 4.760 | 4.000 | 5.520 |
| COUNTRY | e015137 | 7.120 | 2.600 | 2.860 | 4.880 | 6.420 | 0.840 | 4.260 | 6.700 | 7.740 | 3.100 | 2.800 | 7.160 | 7.560 | 5.360 | 3.360 |
| EDANCE | a009068 | 2.400 | 1.940 | 3.840 | 4.800 | 5.280 | 2.320 | 3.760 | 3.940 | 2.680 | 1.940 | 2.360 | 3.520 | 4.640 | 3.540 | 4.520 |
| EDANCE | b001840 | 3.920 | 0.660 | 0.300 | 4.640 | 4.960 | 0.100 | 0.280 | 5.120 | 4.120 | 0.400 | 0.000 | 5.020 | 4.460 | 1.520 | 1.200 |
| EDANCE | b015999 | 7.780 | 2.000 | 1.700 | 6.640 | 6.480 | 0.000 | 6.340 | 8.340 | 8.980 | 0.000 | 0.680 | 7.680 | 8.280 | 7.400 | 7.300 |
| EDANCE | b015503 | 4.200 | 1.200 | 1.000 | 5.300 | 6.500 | 0.200 | 2.900 | 2.900 | 6.400 | 2.300 | 3.400 | 1.000 | 4.200 | 4.600 | 2.300 |
| EDANCE | b019464 | 5.060 | 2.900 | 0.760 | 5.620 | 4.520 | 0.000 | 0.200 | 4.400 | 2.680 | 4.760 | 5.780 | 2.520 | 6.100 | 4.420 | 1.480 |
| EDANCE | b019570 | 5.880 | 0.400 | 4.600 | 7.320 | 6.660 | 0.900 | 6.300 | 7.860 | 6.500 | 3.920 | 3.480 | 8.220 | 7.140 | 6.860 | 5.220 |
| EDANCE | f014939 | 1.880 | 0.820 | 0.860 | 1.220 | 0.660 | 0.260 | 1.360 | 1.880 | 1.980 | 1.480 | 1.200 | 1.560 | 3.020 | 1.880 | 3.020 |
| EDANCE | f008160 | 4.680 | 2.420 | 4.480 | 7.420 | 7.040 | 1.200 | 6.100 | 6.600 | 6.180 | 3.240 | 2.880 | 7.200 | 8.100 | 6.880 | 5.460 |
| EDANCE | f011114 | 3.400 | 3.060 | 4.000 | 5.320 | 5.560 | 1.360 | 4.520 | 2.180 | 2.080 | 3.480 | 4.760 | 1.580 | 5.620 | 2.620 | 2.500 |
| EDANCE | f003748 | 6.800 | 1.660 | 4.240 | 6.540 | 5.840 | 0.260 | 6.560 | 6.380 | 5.140 | 4.860 | 1.160 | 6.800 | 4.720 | 4.980 | 4.800 |
| JAZZ | a003703 | 1.000 | 3.260 | 0.180 | 2.540 | 1.120 | 0.000 | 3.820 | 0.680 | 3.620 | 0.580 | 0.120 | 3.060 | 2.200 | 0.600 | 1.020 |
| JAZZ | e002394 | 1.000 | 0.000 | 1.000 | 1.800 | 1.800 | 3.200 | 4.000 | 6.800 | 3.600 | 1.700 | 1.800 | 3.800 | 4.000 | 0.400 | 2.500 |
| JAZZ | e001113 | 6.980 | 2.120 | 3.480 | 7.640 | 8.360 | 3.520 | 7.400 | 4.560 | 8.220 | 2.320 | 2.940 | 7.780 | 7.120 | 7.100 | 5.180 |
| JAZZ | e004292 | 6.000 | 3.940 | 1.300 | 7.520 | 8.000 | 5.800 | 6.740 | 7.080 | 5.700 | 1.140 | 2.040 | 5.600 | 7.940 | 3.760 | 3.720 |
| JAZZ | e004662 | 2.780 | 0.860 | 2.160 | 1.060 | 3.880 | 0.800 | 4.640 | 2.660 | 4.300 | 1.280 | 0.640 | 3.580 | 6.120 | 1.660 | 2.680 |
| JAZZ | e004944 | 7.600 | 2.860 | 2.740 | 5.220 | 5.640 | 6.600 | 8.320 | 8.520 | 8.660 | 2.560 | 2.900 | 5.280 | 8.540 | 8.060 | 5.080 |
| JAZZ | e004070 | 6.440 | 4.540 | 1.440 | 3.980 | 4.900 | 1.480 | 4.960 | 5.680 | 5.760 | 4.120 | 5.140 | 5.180 | 6.380 | 4.520 | 3.440 |
| JAZZ | e012026 | 2.140 | 4.180 | 0.160 | 4.740 | 5.400 | 1.000 | 2.440 | 3.900 | 2.640 | 3.460 | 0.760 | 1.920 | 7.920 | 2.880 | 6.280 |
| JAZZ | e015744 | 7.540 | 2.120 | 1.560 | 7.860 | 8.120 | 3.440 | 7.300 | 7.900 | 8.140 | 5.360 | 4.900 | 7.700 | 9.080 | 6.560 | 7.180 |
| JAZZ | e015566 | 6.440 | 2.800 | 1.700 | 4.720 | 6.700 | 7.020 | 5.780 | 5.560 | 7.720 | 5.020 | 2.720 | 6.840 | 8.300 | 5.560 | 2.940 |
| METAL | a003208 | 7.440 | 1.620 | 5.800 | 6.280 | 7.220 | 1.100 | 7.880 | 6.980 | 7.000 | 3.880 | 4.260 | 7.840 | 5.400 | 7.820 | 8.120 |
| METAL | b006262 | 5.920 | 3.500 | 3.360 | 6.520 | 4.060 | 0.000 | 5.420 | 2.380 | 3.240 | 1.260 | 1.740 | 5.800 | 6.920 | 3.280 | 5.340 |
| METAL | b007445 | 7.240 | 3.680 | 4.500 | 4.760 | 6.600 | 6.580 | 7.020 | 7.380 | 7.820 | 2.760 | 4.560 | 7.140 | 7.920 | 7.380 | 6.260 |
| METAL | b010346 | 6.600 | 1.780 | 2.340 | 5.980 | 7.060 | 2.520 | 6.820 | 6.160 | 7.260 | 3.800 | 4.800 | 6.720 | 6.940 | 7.320 | 6.960 |
| METAL | b014327 | 1.760 | 0.920 | 3.860 | 6.000 | 3.400 | 0.620 | 5.960 | 4.240 | 2.800 | 1.560 | 2.260 | 5.200 | 5.880 | 4.820 | 5.200 |
| METAL | b017546 | 4.300 | 2.340 | 3.120 | 6.140 | 5.400 | 0.960 | 5.980 | 4.480 | 5.920 | 1.020 | 0.980 | 6.420 | 7.880 | 5.100 | 3.680 |
| METAL | b019571 | 6.760 | 2.000 | 3.720 | 5.460 | 7.540 | 1.380 | 6.300 | 7.600 | 7.380 | 3.160 | 2.200 | 7.700 | 7.000 | 7.080 | 6.080 |
| METAL | f002408 | 6.220 | 0.140 | 5.320 | 5.480 | 6.260 | 1.560 | 7.120 | 5.880 | 6.940 | 2.960 | 8.120 | 8.260 | 7.540 | 5.860 | 6.200 |
| METAL | f000530 | 6.440 | 1.520 | 2.020 | 6.240 | 7.300 | 0.000 | 8.520 | 7.520 | 6.860 | 2.000 | 4.900 | 6.780 | 7.980 | 7.440 | 7.520 |
| METAL | f005072 | 5.640 | 3.420 | 4.360 | 5.060 | 4.800 | 2.700 | 5.140 | 6.640 | 5.540 | 1.920 | 3.740 | 4.760 | 5.760 | 3.620 | 4.380 |
| RAPHIPHOP | a000293 | 7.480 | 3.540 | 2.080 | 7.780 | 8.580 | 0.480 | 7.540 | 7.160 | 5.320 | 2.580 | 3.660 | 8.120 | 9.020 | 2.160 | 5.760 |
| RAPHIPHOP | a000827 | 5.700 | 4.020 | 3.880 | 4.440 | 5.860 | 0.120 | 6.500 | 6.420 | 4.440 | 2.940 | 2.080 | 6.260 | 3.100 | 3.000 | 4.220 |
| RAPHIPHOP | a000897 | 5.940 | 3.560 | 4.120 | 6.020 | 6.580 | 0.400 | 5.500 | 6.740 | 6.760 | 2.000 | 2.160 | 6.480 | 6.860 | 6.220 | 6.400 |
| RAPHIPHOP | a002740 | 6.560 | 2.540 | 3.160 | 6.040 | 7.180 | 0.940 | 5.640 | 6.080 | 6.600 | 2.500 | 2.280 | 7.000 | 6.640 | 5.240 | 6.720 |
| RAPHIPHOP | a009059 | 6.200 | 1.920 | 2.540 | 6.520 | 6.640 | 0.020 | 7.000 | 2.200 | 5.040 | 2.900 | 2.320 | 2.280 | 7.760 | 3.680 | 2.460 |
| RAPHIPHOP | b010144 | 6.380 | 1.820 | 2.100 | 7.120 | 7.500 | 0.080 | 8.180 | 7.280 | 6.700 | 3.520 | 1.860 | 6.700 | 8.920 | 7.820 | 7.880 |
| RAPHIPHOP | b011546 | 5.680 | 3.040 | 0.860 | 5.960 | 7.800 | 0.000 | 6.480 | 7.600 | 5.800 | 2.680 | 1.980 | 7.320 | 5.640 | 5.520 | 6.800 |
| RAPHIPHOP | b010454 | 7.820 | 1.680 | 5.060 | 8.300 | 8.340 | 0.000 | 7.600 | 7.100 | 8.900 | 2.320 | 3.160 | 8.140 | 8.220 | 6.880 | 8.300 |
| RAPHIPHOP | b017461 | 3.660 | 1.780 | 4.280 | 7.500 | 7.100 | 0.180 | 7.840 | 6.900 | 5.780 | 4.220 | 4.100 | 7.320 | 7.700 | 6.600 | 7.160 |
| RAPHIPHOP | b018747 | 7.300 | 0.920 | 0.640 | 8.280 | 8.240 | 0.000 | 8.060 | 7.960 | 7.040 | 1.920 | 0.980 | 7.820 | 8.240 | 7.260 | 7.200 |
| ROCKROLL | b001069 | 6.000 | 2.760 | 5.260 | 5.780 | 3.600 | 4.080 | 6.520 | 3.520 | 6.700 | 1.540 | 3.820 | 6.220 | 7.420 | 5.860 | 6.800 |
| ROCKROLL | b001751 | 3.640 | 1.300 | 0.800 | 2.540 | 1.540 | 0.940 | 0.980 | 1.780 | 3.120 | 1.380 | 2.100 | 3.660 | 2.860 | 3.760 | 4.100 |
| ROCKROLL | b003625 | 1.600 | 2.100 | 3.140 | 5.340 | 3.080 | 0.800 | 3.040 | 4.040 | 2.280 | 1.160 | 1.300 | 4.600 | 4.220 | 5.280 | 5.500 |
| ROCKROLL | b004162 | 5.040 | 3.180 | 3.380 | 3.700 | 4.760 | 1.740 | 5.840 | 4.260 | 5.020 | 2.340 | 2.200 | 4.660 | 5.540 | 5.420 | 4.560 |
| ROCKROLL | b004353 | 6.600 | 1.800 | 3.600 | 7.000 | 6.600 | 1.000 | 5.400 | 4.400 | 5.400 | 4.000 | 2.800 | 4.200 | 8.300 | 6.200 | 5.000 |
| ROCKROLL | b006456 | 5.400 | 1.760 | 8.240 | 7.800 | 5.740 | 0.480 | 6.880 | 6.000 | 6.720 | 1.660 | 3.200 | 8.060 | 6.020 | 7.980 | 6.540 |
| ROCKROLL | b009365 | 5.960 | 1.020 | 4.220 | 4.840 | 7.940 | 0.200 | 4.220 | 3.920 | 6.160 | 2.940 | 3.180 | 6.860 | 7.440 | 7.600 | 5.200 |
| ROCKROLL | b008878 | 5.480 | 2.400 | 3.160 | 5.820 | 4.740 | 0.840 | 4.300 | 5.860 | 4.940 | 2.720 | 2.360 | 4.220 | 4.040 | 3.140 | 1.700 |
| ROCKROLL | b011553 | 1.740 | 0.960 | 1.540 | 2.400 | 2.660 | 0.140 | 3.020 | 3.240 | 3.280 | 0.120 | 0.800 | 1.800 | 3.560 | 2.220 | 1.460 |
| ROCKROLL | e017211 | 2.080 | 1.580 | 1.640 | 1.460 | 2.820 | 1.340 | 1.980 | 1.700 | 2.120 | 1.580 | 1.340 | 2.000 | 2.020 | 2.140 | 1.980 |
| ROMANTIC | d004429 | 8.860 | 6.340 | 3.840 | 7.860 | 8.800 | 7.920 | 7.140 | 8.400 | 8.820 | 5.140 | 3.540 | 8.700 | 9.180 | 8.560 | 8.860 |
| ROMANTIC | d001688 | 4.020 | 0.600 | 1.260 | 2.160 | 3.420 | 2.440 | 2.660 | 3.080 | 3.780 | 0.060 | 0.540 | 3.780 | 3.100 | 2.100 | 2.460 |
| ROMANTIC | d004908 | 5.640 | 2.220 | 0.360 | 8.400 | 7.060 | 5.720 | 5.880 | 5.940 | 5.220 | 1.720 | 0.740 | 6.280 | 7.500 | 4.800 | 6.000 |
| ROMANTIC | d007929 | 4.700 | 1.500 | 2.060 | 6.860 | 7.700 | 7.180 | 5.080 | 3.340 | 6.740 | 0.420 | 2.120 | 6.480 | 6.800 | 4.720 | 4.640 |
| ROMANTIC | d011624 | 6.400 | 0.000 | 1.800 | 3.900 | 6.700 | 3.500 | 5.800 | 3.700 | 6.100 | 0.400 | 0.800 | 7.100 | 7.480 | 0.200 | 3.200 |
| ROMANTIC | d016855 | 4.700 | 3.900 | 0.000 | 3.600 | 5.800 | 4.100 | 4.900 | 4.300 | 4.600 | 0.400 | 0.900 | 4.400 | 4.400 | 4.400 | 4.900 |
| ROMANTIC | d014946 | 7.140 | 2.200 | 1.800 | 6.400 | 6.300 | 7.060 | 2.000 | 5.040 | 7.360 | 0.480 | 2.880 | 5.880 | 7.460 | 5.900 | 4.720 |
| ROMANTIC | d014940 | 5.680 | 2.920 | 0.280 | 6.460 | 7.160 | 4.300 | 6.160 | 5.860 | 7.680 | 1.460 | 0.880 | 6.400 | 7.380 | 3.220 | 4.900 |
| ROMANTIC | d018672 | 2.100 | 0.200 | 1.080 | 0.580 | 2.300 | 1.660 | 1.720 | 2.540 | 2.240 | 0.320 | 0.000 | 2.860 | 2.440 | 2.320 | 1.640 |
| ROMANTIC | d017019 | 3.680 | 3.520 | 1.320 | 1.180 | 3.320 | 3.540 | 4.260 | 4.720 | 2.960 | 2.500 | 1.440 | 3.520 | 4.240 | 3.560 | 3.020 |
BROAD Scores
These are the mean BROAD scores per query assigned by Evalutron graders. The BROAD scores for the 5 candidates returned per algorithm, per query, have been averaged. Values are bounded between 0 (not similar) and 2 (very similar). A perfect score would be 2. Genre labels have been included for reference.
| Genre | Query | ANO | BF1 | BF2 | BSWH1 | BSWH2 | CL1 | CL2 | GT | LR | ME1 | ME2 | PS1 | PS2 | SH1 | SH2 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BAROQUE | d005166 | 1.000 | 0.600 | 0.600 | 1.000 | 1.000 | 0.800 | 1.600 | 1.000 | 1.000 | 0.200 | 0.600 | 1.000 | 1.200 | 1.000 | 0.600 |
| BAROQUE | d007244 | 1.200 | 0.000 | 0.400 | 1.400 | 1.200 | 0.800 | 1.400 | 1.600 | 1.400 | 0.800 | 0.200 | 0.800 | 1.800 | 1.200 | 1.600 |
| BAROQUE | d004490 | 2.000 | 0.800 | 1.600 | 0.800 | 1.600 | 1.200 | 2.000 | 1.800 | 2.000 | 1.200 | 0.800 | 1.800 | 2.000 | 1.800 | 2.000 |
| BAROQUE | d009737 | 2.000 | 1.200 | 0.000 | 0.800 | 2.000 | 1.600 | 1.200 | 0.400 | 1.800 | 0.000 | 0.000 | 2.000 | 2.000 | 0.800 | 0.400 |
| BAROQUE | d009837 | 1.200 | 0.400 | 0.000 | 0.400 | 0.400 | 0.800 | 0.200 | 0.800 | 1.200 | 0.200 | 0.000 | 1.000 | 1.400 | 0.600 | 0.800 |
| BAROQUE | d008174 | 1.200 | 0.800 | 0.400 | 2.000 | 2.000 | 1.800 | 2.000 | 2.000 | 1.600 | 0.600 | 0.200 | 2.000 | 2.000 | 1.200 | 2.000 |
| BAROQUE | d011054 | 1.800 | 1.200 | 0.000 | 1.000 | 1.400 | 1.400 | 1.000 | 1.400 | 1.400 | 0.200 | 0.200 | 1.600 | 1.600 | 1.200 | 1.200 |
| BAROQUE | d018083 | 2.000 | 0.400 | 0.600 | 1.600 | 0.600 | 0.600 | 1.400 | 1.000 | 1.800 | 0.000 | 0.000 | 1.000 | 1.000 | 0.600 | 1.000 |
| BAROQUE | d017335 | 0.400 | 0.000 | 0.000 | 0.000 | 0.600 | 0.000 | 0.400 | 0.600 | 0.200 | 0.200 | 0.000 | 0.000 | 0.400 | 0.400 | 0.200 |
| BAROQUE | d019774 | 1.000 | 0.200 | 0.800 | 1.000 | 1.400 | 1.200 | 1.200 | 1.600 | 1.400 | 0.200 | 1.200 | 2.000 | 2.000 | 1.200 | 1.800 |
| BLUES | e002545 | 1.400 | 1.200 | 0.200 | 1.400 | 1.400 | 0.000 | 1.600 | 1.400 | 1.200 | 0.200 | 0.800 | 1.800 | 2.000 | 1.000 | 1.400 |
| BLUES | e000111 | 0.400 | 1.200 | 0.800 | 0.600 | 0.400 | 0.000 | 1.600 | 1.200 | 0.400 | 0.600 | 1.400 | 1.000 | 1.200 | 1.400 | 1.400 |
| BLUES | e001417 | 1.800 | 0.600 | 0.400 | 1.000 | 1.000 | 1.800 | 2.000 | 2.000 | 1.600 | 1.400 | 0.800 | 1.600 | 1.800 | 1.000 | 1.600 |
| BLUES | e004211 | 0.400 | 0.600 | 0.600 | 0.600 | 1.400 | 1.600 | 1.600 | 0.600 | 0.800 | 0.000 | 0.400 | 1.800 | 2.000 | 1.600 | 0.000 |
| BLUES | e014267 | 1.200 | 0.600 | 0.200 | 1.400 | 0.800 | 0.000 | 1.200 | 1.200 | 0.600 | 0.200 | 0.600 | 1.000 | 1.400 | 0.800 | 0.400 |
| BLUES | e013973 | 1.600 | 0.200 | 0.600 | 1.400 | 1.800 | 0.000 | 2.000 | 2.000 | 1.600 | 0.800 | 0.800 | 1.800 | 1.800 | 2.000 | 1.800 |
| BLUES | e014486 | 0.800 | 0.600 | 0.200 | 1.200 | 1.200 | 0.000 | 0.400 | 1.200 | 0.600 | 0.800 | 0.400 | 1.200 | 1.200 | 1.000 | 1.200 |
| BLUES | e010067 | 0.400 | 0.000 | 0.200 | 1.200 | 1.600 | 0.800 | 1.400 | 0.800 | 0.800 | 1.600 | 0.400 | 0.800 | 2.000 | 0.400 | 0.600 |
| BLUES | e012895 | 1.600 | 0.800 | 0.200 | 1.000 | 1.600 | 1.600 | 1.600 | 1.600 | 1.800 | 0.400 | 0.200 | 1.800 | 2.000 | 1.400 | 1.200 |
| BLUES | e015354 | 1.200 | 0.600 | 0.000 | 1.000 | 1.400 | 0.000 | 1.400 | 1.200 | 1.400 | 0.200 | 0.400 | 0.200 | 1.600 | 1.400 | 2.000 |
| CLASSICAL | d000239 | 1.600 | 1.000 | 0.000 | 1.400 | 1.600 | 1.600 | 1.200 | 2.000 | 1.800 | 0.000 | 0.800 | 2.000 | 2.000 | 1.000 | 1.200 |
| CLASSICAL | d000762 | 1.400 | 0.400 | 0.400 | 1.600 | 1.400 | 1.000 | 1.400 | 1.400 | 1.600 | 0.000 | 0.000 | 2.000 | 2.000 | 1.000 | 1.000 |
| CLASSICAL | d002538 | 1.200 | 0.000 | 0.600 | 1.000 | 1.200 | 0.800 | 1.200 | 1.200 | 1.200 | 0.400 | 0.800 | 1.200 | 1.400 | 1.000 | 1.400 |
| CLASSICAL | d001502 | 2.000 | 0.800 | 1.600 | 2.000 | 2.000 | 1.800 | 2.000 | 2.000 | 2.000 | 1.200 | 1.000 | 2.000 | 2.000 | 2.000 | 2.000 |
| CLASSICAL | d012972 | 1.800 | 0.800 | 0.000 | 1.800 | 2.000 | 1.600 | 1.800 | 2.000 | 1.600 | 0.400 | 0.800 | 2.000 | 2.000 | 1.800 | 1.800 |
| CLASSICAL | d010713 | 1.200 | 0.600 | 0.000 | 1.400 | 1.600 | 1.000 | 1.400 | 1.800 | 1.200 | 0.200 | 0.000 | 1.800 | 1.600 | 0.800 | 1.000 |
| CLASSICAL | d012985 | 2.000 | 0.800 | 0.400 | 0.600 | 1.800 | 1.200 | 1.800 | 1.200 | 2.000 | 0.000 | 1.000 | 2.000 | 1.800 | 1.400 | 2.000 |
| CLASSICAL | d019802 | 1.200 | 0.400 | 0.200 | 0.800 | 1.600 | 0.600 | 1.000 | 0.800 | 0.400 | 0.400 | 0.400 | 1.200 | 1.000 | 1.200 | 1.400 |
| CLASSICAL | d019790 | 1.000 | 1.000 | 1.200 | 1.400 | 1.400 | 0.800 | 1.000 | 1.000 | 1.000 | 0.200 | 0.600 | 1.200 | 1.400 | 1.000 | 1.400 |
| CLASSICAL | d019783 | 1.800 | 0.200 | 0.600 | 1.400 | 1.800 | 1.600 | 1.800 | 1.200 | 1.800 | 0.600 | 0.000 | 1.200 | 2.000 | 2.000 | 1.800 |
| COUNTRY | e002580 | 1.000 | 0.200 | 0.400 | 0.800 | 0.600 | 0.000 | 0.400 | 1.000 | 0.400 | 0.600 | 0.200 | 0.800 | 1.000 | 0.400 | 0.400 |
| COUNTRY | e003090 | 0.600 | 0.200 | 0.600 | 0.400 | 1.000 | 0.000 | 0.800 | 0.800 | 1.000 | 0.400 | 0.200 | 1.000 | 1.600 | 0.800 | 0.800 |
| COUNTRY | e001591 | 0.000 | 0.800 | 0.600 | 0.200 | 1.400 | 0.000 | 1.000 | 1.000 | 0.200 | 0.600 | 0.400 | 1.600 | 1.600 | 1.200 | 1.200 |
| COUNTRY | e006811 | 1.200 | 0.400 | 0.600 | 1.200 | 1.200 | 0.000 | 0.800 | 0.600 | 1.200 | 0.200 | 0.400 | 1.000 | 2.000 | 1.400 | 0.400 |
| COUNTRY | e003155 | 0.000 | 0.000 | 0.200 | 0.000 | 0.600 | 0.000 | 0.800 | 0.400 | 0.400 | 0.000 | 0.000 | 0.800 | 1.200 | 1.200 | 0.600 |
| COUNTRY | e003544 | 1.400 | 0.200 | 0.400 | 1.200 | 0.600 | 0.000 | 1.000 | 0.800 | 0.800 | 0.000 | 0.400 | 1.400 | 1.200 | 1.400 | 1.400 |
| COUNTRY | e014385 | 1.000 | 0.000 | 0.000 | 0.600 | 1.000 | 0.000 | 0.400 | 0.400 | 0.200 | 0.200 | 0.200 | 1.200 | 1.200 | 0.200 | 0.400 |
| COUNTRY | e011855 | 1.600 | 0.000 | 0.000 | 0.600 | 1.800 | 0.000 | 1.000 | 1.000 | 1.400 | 0.200 | 0.000 | 2.000 | 1.200 | 1.400 | 1.200 |
| COUNTRY | e015843 | 1.600 | 0.200 | 0.400 | 1.000 | 0.800 | 0.000 | 1.000 | 1.200 | 1.000 | 0.400 | 1.200 | 1.600 | 1.200 | 0.800 | 1.200 |
| COUNTRY | e015137 | 1.600 | 0.600 | 0.600 | 1.000 | 1.200 | 0.200 | 0.600 | 1.600 | 1.600 | 0.600 | 0.400 | 1.600 | 1.800 | 1.200 | 0.800 |
| EDANCE | a009068 | 0.000 | 0.200 | 0.600 | 1.000 | 1.000 | 0.200 | 0.600 | 0.800 | 0.200 | 0.200 | 0.200 | 0.600 | 0.800 | 0.600 | 0.800 |
| EDANCE | b001840 | 0.400 | 0.000 | 0.000 | 0.600 | 0.600 | 0.000 | 0.000 | 0.800 | 0.800 | 0.000 | 0.000 | 0.800 | 0.600 | 0.000 | 0.000 |
| EDANCE | b015999 | 1.800 | 0.400 | 0.200 | 1.400 | 1.400 | 0.000 | 1.200 | 1.800 | 2.000 | 0.000 | 0.000 | 1.800 | 1.800 | 1.600 | 1.600 |
| EDANCE | b015503 | 0.600 | 0.200 | 0.000 | 1.000 | 1.200 | 0.000 | 0.400 | 0.400 | 1.000 | 0.200 | 0.400 | 0.000 | 0.400 | 0.400 | 0.200 |
| EDANCE | b019464 | 1.200 | 0.600 | 0.000 | 1.600 | 1.000 | 0.000 | 0.000 | 1.000 | 0.600 | 0.800 | 1.400 | 0.600 | 1.600 | 1.000 | 0.200 |
| EDANCE | b019570 | 1.200 | 0.000 | 0.600 | 1.600 | 1.400 | 0.000 | 1.200 | 1.800 | 1.000 | 0.600 | 0.600 | 2.000 | 1.400 | 1.400 | 1.200 |
| EDANCE | f014939 | 0.200 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.200 | 0.000 | 0.000 | 0.000 | 0.400 | 0.000 | 0.600 |
| EDANCE | f008160 | 1.000 | 0.200 | 1.000 | 1.800 | 1.600 | 0.000 | 1.400 | 1.600 | 1.400 | 0.400 | 0.600 | 1.800 | 2.000 | 1.600 | 1.400 |
| EDANCE | f011114 | 0.600 | 0.400 | 0.600 | 1.000 | 1.200 | 0.000 | 1.000 | 0.200 | 0.000 | 0.400 | 0.800 | 0.000 | 1.000 | 0.400 | 0.400 |
| EDANCE | f003748 | 1.600 | 0.200 | 0.800 | 1.600 | 1.200 | 0.000 | 1.200 | 1.200 | 1.200 | 1.000 | 0.200 | 1.400 | 1.000 | 1.000 | 1.000 |
| JAZZ | a003703 | 0.200 | 0.600 | 0.000 | 0.600 | 0.200 | 0.000 | 1.000 | 0.200 | 0.600 | 0.200 | 0.000 | 0.800 | 0.400 | 0.000 | 0.200 |
| JAZZ | e002394 | 0.200 | 0.000 | 0.200 | 0.400 | 0.200 | 0.800 | 1.000 | 1.400 | 0.600 | 0.200 | 0.400 | 0.800 | 1.000 | 0.000 | 0.600 |
| JAZZ | e001113 | 1.400 | 0.400 | 0.800 | 1.800 | 1.800 | 0.800 | 1.600 | 1.000 | 1.800 | 0.400 | 0.400 | 1.600 | 1.600 | 1.600 | 1.000 |
| JAZZ | e004292 | 1.000 | 0.800 | 0.000 | 1.600 | 1.600 | 1.200 | 1.600 | 1.400 | 1.000 | 0.000 | 0.200 | 1.000 | 1.600 | 0.400 | 0.800 |
| JAZZ | e004662 | 0.400 | 0.000 | 0.000 | 0.000 | 0.600 | 0.000 | 0.600 | 0.000 | 0.600 | 0.000 | 0.000 | 0.400 | 1.200 | 0.000 | 0.200 |
| JAZZ | e004944 | 2.000 | 0.400 | 0.400 | 1.200 | 1.200 | 1.600 | 2.000 | 2.000 | 2.000 | 0.200 | 0.600 | 1.200 | 2.000 | 2.000 | 1.000 |
| JAZZ | e004070 | 1.400 | 1.000 | 0.000 | 0.600 | 1.000 | 0.200 | 1.000 | 1.000 | 1.200 | 0.600 | 0.800 | 1.000 | 1.400 | 0.800 | 0.600 |
| JAZZ | e012026 | 0.400 | 0.800 | 0.000 | 1.000 | 1.000 | 0.200 | 0.600 | 0.600 | 0.600 | 0.600 | 0.200 | 0.400 | 2.000 | 0.400 | 1.600 |
| JAZZ | e015744 | 1.400 | 0.200 | 0.200 | 1.800 | 1.800 | 0.400 | 1.200 | 1.600 | 1.800 | 1.000 | 0.800 | 1.800 | 2.000 | 1.200 | 1.200 |
| JAZZ | e015566 | 1.400 | 0.600 | 0.400 | 1.000 | 1.600 | 1.600 | 1.200 | 1.000 | 1.600 | 1.200 | 0.400 | 1.600 | 2.000 | 1.000 | 0.400 |
| METAL | a003208 | 1.400 | 0.000 | 0.800 | 1.000 | 1.200 | 0.000 | 1.800 | 1.400 | 1.200 | 0.400 | 0.400 | 1.600 | 0.800 | 1.600 | 1.800 |
| METAL | b006262 | 1.200 | 0.800 | 1.000 | 1.400 | 0.600 | 0.000 | 1.400 | 0.400 | 0.800 | 0.200 | 0.400 | 1.600 | 1.600 | 0.800 | 1.000 |
| METAL | b007445 | 2.000 | 0.400 | 0.800 | 0.600 | 1.800 | 1.400 | 1.800 | 2.000 | 2.000 | 0.200 | 0.800 | 2.000 | 2.000 | 1.600 | 1.400 |
| METAL | b010346 | 1.600 | 0.000 | 0.000 | 1.400 | 1.800 | 0.200 | 1.400 | 1.400 | 1.800 | 0.400 | 1.000 | 1.600 | 1.800 | 1.800 | 1.400 |
| METAL | b014327 | 0.000 | 0.000 | 0.400 | 0.800 | 0.200 | 0.000 | 0.800 | 0.600 | 0.200 | 0.000 | 0.200 | 0.600 | 1.000 | 0.600 | 0.800 |
| METAL | b017546 | 0.800 | 0.400 | 0.400 | 1.400 | 1.200 | 0.000 | 1.200 | 0.800 | 1.400 | 0.000 | 0.000 | 1.600 | 1.800 | 1.000 | 0.600 |
| METAL | b019571 | 1.600 | 0.200 | 1.000 | 1.200 | 1.800 | 0.200 | 1.400 | 1.600 | 1.400 | 0.400 | 0.400 | 1.600 | 1.600 | 1.400 | 1.400 |
| METAL | f002408 | 1.000 | 0.000 | 1.200 | 1.200 | 1.000 | 0.400 | 1.600 | 1.000 | 1.400 | 0.600 | 1.800 | 1.800 | 1.400 | 1.200 | 1.400 |
| METAL | f000530 | 1.000 | 0.200 | 0.200 | 1.200 | 1.400 | 0.000 | 1.600 | 1.400 | 1.200 | 0.400 | 0.800 | 1.200 | 1.600 | 1.600 | 1.600 |
| METAL | f005072 | 0.600 | 0.400 | 0.600 | 0.800 | 0.600 | 0.200 | 0.800 | 1.200 | 0.600 | 0.000 | 0.400 | 0.400 | 1.000 | 0.400 | 0.200 |
| RAPHIPHOP | a000293 | 1.600 | 0.600 | 0.200 | 1.800 | 2.000 | 0.000 | 1.800 | 1.600 | 1.000 | 0.600 | 0.600 | 2.000 | 2.000 | 0.200 | 1.000 |
| RAPHIPHOP | a000827 | 1.200 | 0.800 | 0.600 | 1.000 | 1.200 | 0.000 | 1.400 | 1.600 | 1.000 | 0.600 | 0.400 | 1.400 | 0.600 | 0.200 | 0.800 |
| RAPHIPHOP | a000897 | 1.600 | 0.600 | 1.000 | 1.400 | 1.800 | 0.000 | 1.400 | 2.000 | 2.000 | 0.200 | 0.200 | 1.800 | 2.000 | 1.600 | 1.400 |
| RAPHIPHOP | a002740 | 1.400 | 0.400 | 0.400 | 1.200 | 1.600 | 0.000 | 1.400 | 1.200 | 1.400 | 0.200 | 0.200 | 1.600 | 1.600 | 1.200 | 1.600 |
| RAPHIPHOP | a009059 | 1.200 | 0.400 | 0.200 | 1.000 | 1.400 | 0.000 | 1.600 | 0.000 | 1.000 | 0.200 | 0.400 | 0.200 | 1.600 | 0.600 | 0.400 |
| RAPHIPHOP | b010144 | 1.200 | 0.400 | 0.200 | 1.400 | 1.800 | 0.000 | 2.000 | 1.800 | 1.400 | 1.000 | 0.200 | 1.600 | 2.000 | 1.800 | 1.800 |
| RAPHIPHOP | b011546 | 1.400 | 0.600 | 0.000 | 1.400 | 1.800 | 0.000 | 1.600 | 1.600 | 1.600 | 0.600 | 0.400 | 2.000 | 1.400 | 1.200 | 1.800 |
| RAPHIPHOP | b010454 | 1.600 | 0.400 | 1.200 | 2.000 | 1.800 | 0.000 | 1.800 | 1.400 | 2.000 | 0.200 | 0.600 | 1.800 | 1.800 | 1.600 | 2.000 |
| RAPHIPHOP | b017461 | 0.800 | 0.400 | 1.000 | 2.000 | 1.800 | 0.000 | 1.800 | 1.800 | 1.400 | 1.000 | 1.000 | 1.800 | 2.000 | 1.600 | 2.000 |
| RAPHIPHOP | b018747 | 1.800 | 0.200 | 0.000 | 2.000 | 2.000 | 0.000 | 2.000 | 2.000 | 2.000 | 0.400 | 0.000 | 2.000 | 2.000 | 1.800 | 1.800 |
| ROCKROLL | b001069 | 1.000 | 0.400 | 0.800 | 1.000 | 0.600 | 0.600 | 1.400 | 0.600 | 1.400 | 0.200 | 0.600 | 1.200 | 1.600 | 1.000 | 1.400 |
| ROCKROLL | b001751 | 1.000 | 0.600 | 0.000 | 0.600 | 0.200 | 0.000 | 0.200 | 0.600 | 0.800 | 0.400 | 0.800 | 0.800 | 0.800 | 0.800 | 1.000 |
| ROCKROLL | b003625 | 0.200 | 0.200 | 0.600 | 1.000 | 0.600 | 0.000 | 0.600 | 0.600 | 0.400 | 0.000 | 0.000 | 0.800 | 1.000 | 1.200 | 1.200 |
| ROCKROLL | b004162 | 1.000 | 0.200 | 0.200 | 0.800 | 1.000 | 0.000 | 1.000 | 1.000 | 0.800 | 0.200 | 0.200 | 1.000 | 1.000 | 0.800 | 0.800 |
| ROCKROLL | b004353 | 1.600 | 0.000 | 0.000 | 1.800 | 1.600 | 0.000 | 1.000 | 0.400 | 0.800 | 0.200 | 0.000 | 0.200 | 2.000 | 1.200 | 0.600 |
| ROCKROLL | b006456 | 1.400 | 0.200 | 2.000 | 1.600 | 1.200 | 0.000 | 1.600 | 1.400 | 1.600 | 0.000 | 0.600 | 2.000 | 1.400 | 2.000 | 1.600 |
| ROCKROLL | b009365 | 1.000 | 0.200 | 1.000 | 1.000 | 2.000 | 0.000 | 0.600 | 0.600 | 1.000 | 0.400 | 0.400 | 1.400 | 1.600 | 1.600 | 1.000 |
| ROCKROLL | b008878 | 1.400 | 0.200 | 0.400 | 1.400 | 1.000 | 0.000 | 0.800 | 1.400 | 1.000 | 0.400 | 0.200 | 1.000 | 0.800 | 0.400 | 0.200 |
| ROCKROLL | b011553 | 0.400 | 0.400 | 0.400 | 0.600 | 0.800 | 0.000 | 0.800 | 0.800 | 0.800 | 0.000 | 0.200 | 0.400 | 0.800 | 0.600 | 0.200 |
| ROCKROLL | e017211 | 0.000 | 0.000 | 0.000 | 0.000 | 0.200 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
| ROMANTIC | d004429 | 2.000 | 1.400 | 0.600 | 1.800 | 2.000 | 1.800 | 1.600 | 2.000 | 2.000 | 1.000 | 0.600 | 2.000 | 2.000 | 2.000 | 2.000 |
| ROMANTIC | d001688 | 0.600 | 0.000 | 0.000 | 0.200 | 0.400 | 0.200 | 0.400 | 0.200 | 0.600 | 0.000 | 0.000 | 0.600 | 0.200 | 0.000 | 0.000 |
| ROMANTIC | d004908 | 1.200 | 0.400 | 0.000 | 2.000 | 1.600 | 1.200 | 1.400 | 1.400 | 1.200 | 0.400 | 0.200 | 1.400 | 1.600 | 0.800 | 1.200 |
| ROMANTIC | d007929 | 1.000 | 0.400 | 0.400 | 1.600 | 1.400 | 1.600 | 1.000 | 1.000 | 1.600 | 0.000 | 0.600 | 1.400 | 1.600 | 1.200 | 1.000 |
| ROMANTIC | d011624 | 1.600 | 0.000 | 0.400 | 1.000 | 1.800 | 0.800 | 1.200 | 1.000 | 1.600 | 0.000 | 0.000 | 1.600 | 1.800 | 0.000 | 0.600 |
| ROMANTIC | d016855 | 0.800 | 0.800 | 0.000 | 1.000 | 1.400 | 0.800 | 1.200 | 1.000 | 1.000 | 0.000 | 0.200 | 0.800 | 1.000 | 0.800 | 1.000 |
| ROMANTIC | d014946 | 1.400 | 0.400 | 0.400 | 1.400 | 1.400 | 1.600 | 0.400 | 1.000 | 1.600 | 0.000 | 0.600 | 1.400 | 1.600 | 1.400 | 1.200 |
| ROMANTIC | d014940 | 1.200 | 0.400 | 0.000 | 1.600 | 1.600 | 0.800 | 1.600 | 1.200 | 1.800 | 0.200 | 0.200 | 1.800 | 1.800 | 0.600 | 1.000 |
| ROMANTIC | d018672 | 0.200 | 0.000 | 0.200 | 0.000 | 0.200 | 0.000 | 0.000 | 0.200 | 0.200 | 0.000 | 0.000 | 0.400 | 0.200 | 0.400 | 0.000 |
| ROMANTIC | d017019 | 0.400 | 0.600 | 0.000 | 0.000 | 0.400 | 0.600 | 0.600 | 0.800 | 0.200 | 0.200 | 0.000 | 0.400 | 0.600 | 0.000 | 0.200 |
Raw Scores
The raw data derived from the Evalutron 6000 human evaluations are located on the 2009:Audio Music Similarity and Retrieval Raw Data page.
Metadata and Distance Space Evaluation
The following reports provide evaluation statistics based on analysis of the distance space and metadata matches and include:
- Neighbourhood clustering by artist, album and genre
- Artist-filtered genre clustering
- How often the triangular inequality holds
- Statistics on 'hubs' (tracks similar to many tracks) and orphans (tracks that are not similar to any other tracks at N results).
Reports
ANO = Anonymous
BF1 = Benjamin Fields (chr12)
BF2 = Benjamin Fields (mfcc10)
BSWH1 = Dmitry Bogdanov, Joan Serrà, Nicolas Wack, and Perfecto Herrera (clas)
BSWH2 = Dmitry Bogdanov, Joan Serrà, Nicolas Wack, and Perfecto Herrera (hybrid)
CL1 = Chuan Cao, Ming Li
CL2 = Chuan Cao, Ming Li
GT = George Tzanetakis
LR = Thomas Lidy, Andreas Rauber]
ME1 = François Maillet, Douglas Eck (mlp)
ME2 = François Maillet, Douglas Eck (sda)
PS1 = Tim Pohle, Dominik Schnitzer (2007)
PS2 = Tim Pohle, Dominik Schnitzer (2009)
SH1 = Stephan Hübler
SH2 = Stephan Hübler
Run Times
| Participant | Machine | Runtime (dd:hh:mm) |
|---|---|---|
| ANO | ALE | Feat/Dist 00:00:22/00:00:04 |
| BF1 | ALE | 10:00:00 |
| BF2 | ALE | 10:00:00 |
| BSWH1 | ALE | Feat/Dist 00:20:06/00:00:22 |
| BSWH2 | ALE | Feat/Dist 00:20:06/00:01:12 |
| CL1 | FAST3 | Lost - HD Failure |
| CL2 | FAST3 | Lost - HD Failure |
| GT | ALE | Feat/Dist 00:00:23/00:00:20 |
| LR | ALE | Feat/Dist 00:05:09/00:02:05 |
| ME1 | ALE | Feat/Dist 00:16:10/00:12:37 |
| ME2 | ALE | Feat/Dist 00:16:08/02:23:27 |
| PS1 | ALE | Feat/Dist 00:00:42/00:03:51 |
| PS2 | ALE | Feat/Dist 00:03:40/00:05:56 |
| SH1 | ALE | 00:04:06 |
| SH2 | ALE | 00:05:50 |
