2007:Audio Music Similarity and Retrieval Results

Introduction

These are the results for the 2007 running of the Audio Music Similarity and Retrieval task set. For background information about this task set please refer to the 2007:Audio Music Similarity and Retrieval page.

Each system was given 7000 songs chosen from IMIRSEL's "uspop", "uscrap" and "american" "classical" and "sundry" collections. Each system then returned a 7000x7000 distance matrix. 100 songs were randomly selected from the 10 genre groups (10 per genre) as queries and the first 5 most highly ranked songs out of the 7000 were extracted for each query (after filtering out the query itself, returned results from the same artist were also omitted). Then, for each query, the returned results (candidates) from all participants were grouped and were evaluated by human graders using the Evalutron 6000 grading system. Each individual query/candidate set was evaluated by a single grader. For each query/candidate pair, graders provided two scores. Graders were asked to provide 1 categorical score with 3 categories: NS,SS,VS as explained below, and one fine score (in the range from 0 to 10). A description and analysis is provided below.

The systems read in 30 second audio clips as their raw data. The same 30 second clips were used in the grading stage.

Summary Data on Human Evaluations (Evalutron 6000)

Number of evaluators = 20
Number of evaluations per query/candidate pair = 1
Number of queries per grader = 5
Size of the candidate lists = 48.32
Number of randomly selected queries = 100
Number of query/candidate pairs graded = 4832

General Legend

Team ID

BK1 = Klaas Bosteels, Etienne E. Kerre 1
BK2 = Klaas Bosteels, Etienne E. Kerre 2
CB1 = Christoph Bastuck 1
CB2 = Christoph Bastuck 2
CB3 = Christoph Bastuck 3
GT = George Tzanetakis
LB = Luke Barrington, Douglas Turnbull, David Torres, Gert Lanskriet
ME = Michael I. Mandel, Daniel P. W. Ellis
PC = Aliaksandr Paradzinets, Liming Chen
PS = Tim Pohle, Dominik Schnitzer
TL1 = Thomas Lidy, Andreas Rauber, Antonio Pertusa, José Manuel Iñesta 1
TL2 = Thomas Lidy, Andreas Rauber, Antonio Pertusa, José Manuel Iñesta 2

Broad Categories

NS = Not Similar
SS = Somewhat Similar
VS = Very Similar

Calculating Summary Measures

Fine⁽¹⁾ = Sum of fine-grained human similarity decisions (0-10).
PSum⁽¹⁾ = Sum of human broad similarity decisions: NS=0, SS=1, VS=2.
WCsum⁽¹⁾ = 'World Cup' scoring: NS=0, SS=1, VS=3 (rewards Very Similar).
SDsum⁽¹⁾ = 'Stephen Downie' scoring: NS=0, SS=1, VS=4 (strongly rewards Very Similar).
Greater0⁽¹⁾ = NS=0, SS=1, VS=1 (binary relevance judgement).
Greater1⁽¹⁾ = NS=0, SS=0, VS=1 (binary relevance judgement using only Very Similar).

⁽¹⁾Normalized to the range 0 to 1.

Overall Summary Results

NB: The results for BK2 were interpolated from partial data due to a runtime error.

	BK1	BK2	CB1	CB2	CB3	GT	LB	ME	PC	PS	TL1	TL2
Fine	0.412	0.178	0.539	0.446	0.439	0.554	0.541	0.512	0.377	0.568	0.519	0.491
Psum	0.417	0.147	0.586	0.468	0.457	0.597	0.582	0.539	0.376	0.619	0.556	0.521
WCsum	0.361	0.115	0.528	0.414	0.407	0.534	0.520	0.480	0.325	0.560	0.491	0.459
SDsum	0.333	0.099	0.499	0.387	0.382	0.503	0.489	0.451	0.299	0.531	0.459	0.428
Greater0	0.586	0.244	0.760	0.630	0.608	0.786	0.768	0.716	0.530	0.796	0.750	0.708
Greater1	0.246	0.050	0.412	0.304	0.306	0.408	0.396	0.362	0.222	0.442	0.362	0.334

download these results as csv

Friedman Test with Multiple Comparisons Results (p=0.05)

The Friedman test was run in MATLAB against the Fine summary data over the 100 queries.
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

Friedman's ANOVA Table
Source	SS	df	MS	Chi-sq	Prob>Chi-sq
Columns	3.0110e+003	11	273.7314	232.0407	0
Error	1.1263e+004	1089	10.3425
Total	1.4274e+004	1199

download these results as csv

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
LB	CB1	-1.7998	-0.1350	1.5298	FALSE
LB	CB2	0.1302	1.7950	3.4598	TRUE
LB	CB3	0.1002	1.7650	3.4298	TRUE
LB	BK1	-0.0598	1.6050	3.2698	FALSE
LB	BK2	3.4352	5.1000	6.7648	TRUE
LB	TL1	-1.4748	0.1900	1.8548	FALSE
LB	TL2	-0.9048	0.7600	2.4248	FALSE
LB	ME	-0.9098	0.7550	2.4198	FALSE
LB	PC	1.2052	2.8700	4.5348	TRUE
LB	PS	-2.3198	-0.6550	1.0098	FALSE
LB	GT	-2.2748	-0.6100	1.0548	FALSE
CB1	CB2	0.2652	1.9300	3.5948	TRUE
CB1	CB3	0.2352	1.9000	3.5648	TRUE
CB1	BK1	0.0752	1.7400	3.4048	TRUE
CB1	BK2	3.5702	5.2350	6.8998	TRUE
CB1	TL1	-1.3398	0.3250	1.9898	FALSE
CB1	TL2	-0.7698	0.8950	2.5598	FALSE
CB1	ME	-0.7748	0.8900	2.5548	FALSE
CB1	PC	1.3402	3.0050	4.6698	TRUE
CB1	PS	-2.1848	-0.5200	1.1448	FALSE
CB1	GT	-2.1398	-0.4750	1.1898	FALSE
CB2	CB3	-1.6948	-0.0300	1.6348	FALSE
CB2	BK1	-1.8548	-0.1900	1.4748	FALSE
CB2	BK2	1.6402	3.3050	4.9698	TRUE
CB2	TL1	-3.2698	-1.6050	0.0598	FALSE
CB2	TL2	-2.6998	-1.0350	0.6298	FALSE
CB2	ME	-2.7048	-1.0400	0.6248	FALSE
CB2	PC	-0.5898	1.0750	2.7398	FALSE
CB2	PS	-4.1148	-2.4500	-0.7852	TRUE
CB2	GT	-4.0698	-2.4050	-0.7402	TRUE
CB3	BK1	-1.8248	-0.1600	1.5048	FALSE
CB3	BK2	1.6702	3.3350	4.9998	TRUE
CB3	TL1	-3.2398	-1.5750	0.0898	FALSE
CB3	TL2	-2.6698	-1.0050	0.6598	FALSE
CB3	ME	-2.6748	-1.0100	0.6548	FALSE
CB3	PC	-0.5598	1.1050	2.7698	FALSE
CB3	PS	-4.0848	-2.4200	-0.7552	TRUE
CB3	GT	-4.0398	-2.3750	-0.7102	TRUE
BK1	BK2	1.8302	3.4950	5.1598	TRUE
BK1	TL1	-3.0798	-1.4150	0.2498	FALSE
BK1	TL2	-2.5098	-0.8450	0.8198	FALSE
BK1	ME	-2.5148	-0.8500	0.8148	FALSE
BK1	PC	-0.3998	1.2650	2.9298	FALSE
BK1	PS	-3.9248	-2.2600	-0.5952	TRUE
BK1	GT	-3.8798	-2.2150	-0.5502	TRUE
BK2	TL1	-6.5748	-4.9100	-3.2452	TRUE
BK2	TL2	-6.0048	-4.3400	-2.6752	TRUE
BK2	ME	-6.0098	-4.3450	-2.6802	TRUE
BK2	PC	-3.8948	-2.2300	-0.5652	TRUE
BK2	PS	-7.4198	-5.7550	-4.0902	TRUE
BK2	GT	-7.3748	-5.7100	-4.0452	TRUE
TL1	TL2	-1.0948	0.5700	2.2348	FALSE
TL1	ME	-1.0998	0.5650	2.2298	FALSE
TL1	PC	1.0152	2.6800	4.3448	TRUE
TL1	PS	-2.5098	-0.8450	0.8198	FALSE
TL1	GT	-2.4648	-0.8000	0.8648	FALSE
TL2	ME	-1.6698	-0.0050	1.6598	FALSE
TL2	PC	0.4452	2.1100	3.7748	TRUE
TL2	PS	-3.0798	-1.4150	0.2498	FALSE
TL2	GT	-3.0348	-1.3700	0.2948	FALSE
ME	PC	0.4502	2.1150	3.7798	TRUE
ME	PS	-3.0748	-1.4100	0.2548	FALSE
ME	GT	-3.0298	-1.3650	0.2998	FALSE
PC	PS	-5.1898	-3.5250	-1.8602	TRUE
PC	GT	-5.1448	-3.4800	-1.8152	TRUE
PS	GT	-1.6198	0.0450	1.7098	FALSE

download these results as csv

Summary Results by Query

These are the mean FINE scores per query assigned by Evalutron graders. The FINE scores for the 5 candidates returned per algorithm, per query, have been averaged. Values are bounded between 0.0 and 10.0. A perfect score would be 10. Genre labels have been included for reference.

Genre	Query	BK1	BK2	CB1	CB2	CB3	GT	LB	ME	PC	PS	TL1	TL2
BAROQUE	d004553	1.500	2.920	8.500	7.740	8.300	8.920	8.740	8.960	7.420	8.800	6.960	5.880
BAROQUE	d006674	7.740	2.400	9.300	7.840	9.420	9.380	9.000	9.280	7.060	9.200	9.700	9.680
BAROQUE	d008942	6.400	1.000	7.000	5.400	7.800	7.200	8.400	5.800	2.600	6.980	6.480	6.980
BAROQUE	d010361	0.340	1.640	6.880	5.420	6.940	5.640	5.300	6.700	4.840	8.880	8.360	7.680
BAROQUE	d011850	9.040	1.760	8.920	8.100	7.900	9.080	8.760	7.560	2.980	9.060	9.020	9.380
BAROQUE	d012288	0.360	2.280	8.900	2.740	5.660	7.140	8.600	7.800	4.400	8.760	8.400	9.160
BAROQUE	d017287	1.060	1.620	9.480	8.900	8.180	9.600	9.140	9.220	5.260	7.800	8.840	8.660
BAROQUE	d018106	0.340	0.900	6.000	3.560	6.720	8.020	7.300	7.660	7.240	7.800	5.860	4.720
BAROQUE	d018433	1.680	2.200	9.020	7.000	7.680	9.360	8.900	9.420	6.360	7.640	8.880	8.660
BAROQUE	d018533	0.120	1.940	7.140	6.660	6.340	7.440	7.700	6.720	6.160	7.480	6.300	7.220
BLUES	e000039	6.160	5.080	2.660	2.780	1.400	5.060	5.200	5.700	5.600	4.880	5.820	2.340
BLUES	e001388	5.880	3.960	4.000	2.320	6.700	8.180	7.020	7.780	8.460	7.920	5.380	5.380
BLUES	e004743	8.800	7.100	5.500	2.800	0.700	6.900	2.800	8.400	0.900	5.600	4.800	4.800
BLUES	e004899	3.300	3.120	3.000	2.820	1.380	3.100	5.360	5.700	5.440	3.120	2.460	1.500
BLUES	e006420	1.960	1.120	3.600	3.120	1.860	3.140	3.260	2.220	2.300	2.960	0.400	0.820
BLUES	e011446	6.860	5.800	7.980	8.200	8.580	7.560	7.440	6.260	6.420	7.100	7.340	6.800
BLUES	e012584	4.040	1.540	1.480	2.220	1.660	3.840	2.240	1.160	2.200	4.200	4.780	2.620
BLUES	e013173	5.200	5.320	4.920	1.500	1.760	5.180	7.340	6.340	7.880	7.100	7.060	4.680
BLUES	e015590	3.000	2.560	4.780	2.600	2.620	3.680	3.440	3.720	1.320	1.700	2.620	2.480
BLUES	e018053	1.360	2.120	4.560	2.460	1.380	3.080	4.720	0.760	2.980	3.980	2.720	1.980
CLASSICAL	d000429	4.640	1.960	4.780	3.280	4.840	5.820	5.900	5.140	0.360	5.840	4.780	4.880
CLASSICAL	d003313	1.020	2.880	7.480	8.120	5.940	4.360	7.420	6.380	5.200	7.580	5.740	5.980
CLASSICAL	d011706	3.280	3.240	9.040	8.940	9.060	8.920	9.020	9.100	1.680	8.140	7.920	5.460
CLASSICAL	d012929	0.760	1.760	8.000	7.940	7.720	7.820	7.240	7.540	7.340	8.060	7.660	7.380
CLASSICAL	d014060	1.320	2.660	7.500	6.860	6.740	7.320	8.360	6.520	5.440	7.040	6.560	6.420
CLASSICAL	d014334	1.100	2.600	9.200	9.200	8.860	8.800	8.000	8.100	5.400	8.160	8.200	8.200
CLASSICAL	d015555	5.220	0.000	5.100	0.400	1.940	5.140	4.900	3.900	0.100	6.940	5.440	4.580
CLASSICAL	d017438	0.000	0.740	2.900	2.640	4.480	5.700	4.380	6.940	3.460	4.080	3.220	1.900
CLASSICAL	d017826	0.000	1.380	2.080	1.140	2.640	4.000	1.860	4.800	1.420	3.980	3.700	2.120
CLASSICAL	d019684	0.120	0.500	8.380	7.400	8.200	7.240	6.940	7.560	5.700	7.420	8.340	8.560
COUNTRY	b008293	7.200	1.800	7.800	3.000	3.200	5.200	7.000	5.000	3.000	4.200	7.000	7.000
COUNTRY	e003167	2.940	3.240	5.880	4.140	6.140	7.460	5.160	3.360	2.260	3.460	3.340	2.120
COUNTRY	e005922	1.240	1.320	2.540	0.360	0.240	3.580	0.860	1.820	1.900	1.400	2.360	2.120
COUNTRY	e008013	5.920	2.980	7.100	7.480	6.240	5.320	5.720	2.400	5.180	5.920	6.620	5.200
COUNTRY	e008637	6.340	2.960	5.460	3.220	0.780	4.220	3.580	6.040	1.400	5.820	5.980	6.480
COUNTRY	e009438	5.600	1.140	4.020	3.120	3.420	3.520	4.120	5.780	2.960	7.240	4.660	3.520
COUNTRY	e011578	6.620	3.580	6.360	4.220	2.100	1.620	5.500	4.340	5.080	3.620	1.720	1.940
COUNTRY	e011702	0.220	0.920	0.380	1.560	0.000	0.040	0.000	0.440	7.140	0.120	0.000	1.760
COUNTRY	e014903	5.340	2.200	4.960	1.900	2.400	3.500	2.840	1.340	4.320	2.740	3.980	3.320
COUNTRY	e018325	5.080	3.240	4.760	4.020	3.100	5.720	4.280	3.120	2.340	4.200	4.300	2.800
EDANCE	a002630	4.840	0.100	3.460	2.800	3.300	3.360	6.220	5.200	2.880	3.760	5.480	4.720
EDANCE	a003317	6.200	0.480	5.280	5.660	2.640	7.300	3.880	3.520	4.360	6.860	5.320	6.480
EDANCE	a003588	8.780	0.000	5.400	6.340	6.840	6.040	4.380	4.000	1.560	9.100	6.720	2.120
EDANCE	a006326	1.800	0.500	1.540	2.140	3.840	1.900	2.720	3.580	1.500	3.140	1.840	2.500
EDANCE	b007008	7.420	0.000	4.420	4.980	5.960	8.000	7.180	8.200	5.960	8.400	6.020	5.620
EDANCE	b008903	5.020	0.480	1.620	2.760	5.020	5.260	4.580	6.900	5.480	5.500	6.760	5.600
EDANCE	b009057	1.940	0.060	4.820	4.360	5.360	2.780	5.020	3.240	1.260	3.000	2.840	6.840
EDANCE	f000987	0.200	0.600	3.700	2.100	2.100	2.600	3.700	5.400	3.300	5.700	0.200	0.400
EDANCE	f005146	2.540	1.000	5.320	3.740	2.280	3.320	3.580	2.740	1.360	6.600	2.620	6.500
EDANCE	f010067	7.000	0.800	5.400	4.400	6.200	4.800	4.600	6.000	7.000	5.400	5.200	5.200
JAZZ	e001054	4.000	4.000	7.000	6.200	6.400	6.600	5.400	4.800	5.400	5.800	5.600	5.600
JAZZ	e004497	6.120	5.900	3.620	2.760	1.520	6.420	6.020	5.520	1.840	5.020	5.780	6.000
JAZZ	e005048	8.240	7.100	4.060	4.140	1.300	4.200	2.300	6.600	7.920	3.120	4.900	3.420
JAZZ	e007252	1.400	1.140	2.060	3.140	2.700	1.180	1.000	4.680	2.320	5.380	2.900	2.520
JAZZ	e007772	5.540	4.300	3.440	3.640	3.160	2.920	1.920	1.820	2.660	3.780	2.580	4.140
JAZZ	e007968	6.600	4.000	6.100	4.640	4.540	6.940	6.200	7.880	7.340	7.600	7.000	7.340
JAZZ	e009441	5.360	0.720	4.200	1.800	0.380	1.500	4.760	2.800	3.440	4.540	4.260	3.280
JAZZ	e010478	6.200	5.680	3.020	3.900	6.040	3.820	2.340	4.820	6.220	3.700	3.460	4.680
JAZZ	e016650	3.780	2.860	6.600	4.040	0.000	4.280	4.320	5.100	3.380	5.080	0.940	0.940
JAZZ	e019093	7.140	6.340	9.040	5.800	4.060	4.900	8.880	5.120	6.840	7.960	6.520	5.940
METAL	b001148	0.920	0.640	5.180	4.920	6.140	4.880	6.720	6.840	2.280	5.700	5.360	5.480
METAL	b002842	6.560	0.080	6.680	5.680	2.560	7.540	5.340	5.320	5.480	6.360	4.400	2.780
METAL	b005320	6.060	1.100	6.080	3.140	2.660	5.440	7.000	6.880	1.420	6.020	5.380	6.300
METAL	b005832	3.380	0.480	6.780	4.120	4.140	4.180	6.360	6.040	4.240	3.440	4.900	5.600
METAL	b006870	4.960	0.360	6.800	5.720	6.220	6.360	5.520	7.260	3.800	6.180	7.100	6.760
METAL	b010213	0.960	0.560	1.020	2.020	2.060	3.500	1.140	0.200	1.660	2.400	2.200	2.000
METAL	b010569	6.200	2.200	6.400	5.400	3.200	5.400	6.800	5.200	3.000	4.200	6.200	4.800
METAL	b011148	7.860	0.300	6.740	6.300	7.200	5.920	7.640	3.320	5.080	6.420	6.960	7.620
METAL	b011701	6.460	0.000	6.660	7.220	1.640	7.400	7.940	8.160	5.100	8.180	8.160	7.760
METAL	f012092	6.560	0.440	7.280	8.260	6.780	7.320	7.900	4.360	1.040	5.600	6.740	5.020
RAPHIPHOP	a000154	8.660	0.360	9.160	9.120	9.340	8.880	8.900	8.360	8.840	8.820	9.400	7.680
RAPHIPHOP	a000895	7.200	1.340	6.820	7.200	6.620	7.140	7.540	6.780	5.320	7.880	7.140	7.300
RAPHIPHOP	a001600	2.440	3.760	1.400	0.420	0.220	2.840	5.660	4.120	1.240	1.640	2.360	2.260
RAPHIPHOP	a003760	6.660	0.420	7.200	6.040	7.080	7.620	7.280	4.640	1.460	7.140	6.720	7.320
RAPHIPHOP	a004471	6.480	0.040	7.600	7.480	7.640	7.540	7.040	7.000	7.460	7.740	8.480	7.840
RAPHIPHOP	a006304	5.500	0.000	5.420	6.300	5.980	5.920	6.240	4.040	0.200	6.580	4.340	6.280
RAPHIPHOP	a008309	8.380	0.620	7.120	6.420	8.280	8.380	7.500	7.500	1.100	7.640	7.980	7.660
RAPHIPHOP	b000472	6.060	0.000	5.820	2.740	6.240	6.720	4.880	6.080	5.400	4.400	6.340	3.300
RAPHIPHOP	b001007	3.960	0.840	2.700	3.600	4.660	3.540	2.920	5.700	1.760	2.140	1.200	1.040
RAPHIPHOP	b011158	7.840	0.800	8.140	9.080	8.820	8.900	7.360	7.920	8.480	8.940	8.940	8.900
ROCKROLL	b000264	0.760	0.000	0.100	0.720	0.280	0.940	0.360	0.160	0.480	0.200	0.920	0.320
ROCKROLL	b000459	2.640	1.100	3.900	4.520	3.180	3.780	3.060	4.780	1.540	2.300	5.400	3.780
ROCKROLL	b001686	2.600	0.000	4.200	6.500	0.900	5.700	2.700	5.700	0.300	3.800	2.900	1.400
ROCKROLL	b005863	1.740	0.220	4.600	1.120	1.460	2.900	2.500	0.200	1.380	1.300	2.700	2.200
ROCKROLL	b008067	4.000	1.200	4.800	2.000	1.600	4.800	5.200	4.600	2.400	3.800	5.800	5.000
ROCKROLL	b009038	2.640	0.060	1.980	2.700	2.020	2.120	3.060	0.820	0.400	2.140	2.200	2.400
ROCKROLL	b012623	1.920	0.000	3.940	0.880	0.200	3.100	2.020	1.940	2.080	2.020	1.500	2.080
ROCKROLL	b015975	4.560	0.760	5.820	5.980	3.540	4.460	6.100	3.600	2.740	5.940	6.540	5.540
ROCKROLL	b016172	1.900	1.040	3.800	2.100	1.900	3.960	3.860	1.240	2.240	2.560	3.460	2.920
ROCKROLL	b019890	3.420	0.480	4.520	1.960	1.060	3.380	2.300	1.080	3.360	3.560	3.760	3.820
ROMANTIC	d000095	1.600	2.340	7.640	6.360	7.740	7.420	6.860	7.040	4.200	7.740	3.300	3.020
ROMANTIC	d000491	0.980	2.440	5.080	5.020	7.280	8.060	6.920	6.120	1.320	8.960	7.160	5.340
ROMANTIC	d003906	6.800	1.560	4.600	3.960	4.760	8.020	5.840	7.020	4.200	7.300	6.740	7.140
ROMANTIC	d005430	2.200	1.100	2.100	0.960	5.260	7.680	4.600	7.200	7.480	8.780	1.880	1.880
ROMANTIC	d008298	3.860	0.260	5.120	1.840	3.000	5.660	5.200	2.600	0.000	7.120	5.000	6.260
ROMANTIC	d008438	2.160	3.100	7.520	7.020	8.560	7.240	7.200	8.320	7.660	7.840	7.460	7.020
ROMANTIC	d009148	6.260	2.360	3.960	4.220	5.380	5.940	5.780	2.960	1.240	4.760	6.520	6.500
ROMANTIC	d009455	2.660	2.080	3.220	4.620	2.480	6.240	4.900	2.160	6.180	8.360	5.080	6.140
ROMANTIC	d010229	7.000	0.020	6.300	4.720	4.700	7.100	4.700	0.000	1.420	6.900	4.900	5.700
ROMANTIC	d010380	0.000	0.380	7.280	5.340	1.700	4.700	7.120	4.500	0.440	8.380	4.840	5.640

download these results as csv

These are the mean BROAD scores per query assigned by Evalutron graders. The BROAD scores for the 5 candidates returned per algorithm, per query, have been averaged. Values are bounded between 0 (not similar) and 2 (very similar). A perfect score would be 2. Genre labels have been included for reference.

Genre	Query	BK1	BK2	CB1	CB2	CB3	GT	LB	ME	PC	PS	TL1	TL2
BAROQUE	d004553	1.6	0.0	1.2	1.4	0.4	1.8	0.4	0.6	0.6	1.8	1.4	1.8
BAROQUE	d006674	0.4	0.6	0.2	0.0	0.0	0.6	1.2	0.8	0.2	0.2	0.4	0.4
BAROQUE	d008942	1.6	0.0	1.4	1.8	1.8	1.8	1.8	1.0	0.0	2.0	1.0	1.6
BAROQUE	d010361	0.2	0.0	0.0	0.2	0.6	0.0	0.4	0.4	0.2	0.4	0.2	0.2
BAROQUE	d011850	1.8	0.0	2.0	2.0	2.0	1.6	2.0	2.0	1.6	2.0	1.8	1.8
BAROQUE	d012288	0.8	0.0	0.4	0.2	0.4	0.6	1.4	1.0	0.4	0.4	1.0	1.0
BAROQUE	d017287	1.6	0.0	2.0	1.4	1.8	2.0	1.8	1.2	0.2	1.8	1.6	2.0
BAROQUE	d018106	2.0	0.0	1.0	1.0	1.4	1.2	0.8	0.8	0.4	2.0	1.4	0.4
BAROQUE	d018433	1.2	0.0	1.4	1.4	1.4	1.4	1.2	1.2	1.6	1.4	1.8	1.4
BAROQUE	d018533	2.0	0.0	2.0	2.0	2.0	2.0	2.0	2.0	2.0	2.0	2.0	1.6
BLUES	e000039	0.8	0.0	0.4	0.8	0.8	0.4	0.4	1.2	0.2	0.2	0.0	0.0
BLUES	e001388	2.0	0.0	1.8	1.6	2.0	2.0	2.0	2.0	0.0	1.8	1.8	1.8
BLUES	e004743	0.4	0.0	1.2	0.6	0.8	0.8	1.2	1.2	1.0	0.6	1.0	1.2
BLUES	e004899	0.0	0.0	1.0	1.0	1.0	1.0	1.2	1.2	0.4	1.0	1.0	1.0
BLUES	e006420	1.0	0.0	1.0	0.4	0.4	1.0	1.2	1.2	0.0	1.0	1.0	1.0
BLUES	e011446	1.4	0.0	1.4	1.4	0.6	1.4	1.0	1.0	1.0	1.2	0.8	0.6
BLUES	e012584	0.6	0.0	0.6	1.0	0.8	0.8	1.0	1.0	0.6	0.4	1.0	1.2
BLUES	e013173	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
BLUES	e015590	0.4	0.0	0.8	1.6	0.2	1.2	0.4	1.4	0.0	0.6	0.8	0.2
BLUES	e018053	1.4	0.0	1.2	0.4	1.4	1.4	1.2	1.6	1.2	1.0	1.4	0.8
CLASSICAL	d000429	0.8	0.0	0.4	0.6	0.4	0.6	0.8	0.2	0.0	0.4	0.8	0.6
CLASSICAL	d003313	1.4	0.4	1.6	1.4	0.8	1.4	1.8	1.2	0.6	1.0	1.6	1.2
CLASSICAL	d011706	0.4	0.0	1.2	0.8	1.4	0.6	1.2	0.6	0.0	0.6	0.6	1.8
CLASSICAL	d012929	0.2	0.0	1.2	0.0	0.2	0.6	0.6	0.0	0.2	0.4	0.6	0.6
CLASSICAL	d014060	1.2	0.0	0.2	0.4	1.2	1.2	1.2	1.6	1.2	1.4	1.8	1.2
CLASSICAL	d014334	0.0	0.0	0.0	0.2	0.0	0.4	0.0	0.0	0.0	0.2	0.0	0.0
CLASSICAL	d015555	1.0	0.0	1.8	1.2	1.4	1.4	1.0	1.6	0.4	1.4	1.8	1.6
CLASSICAL	d017438	1.6	0.0	0.8	1.0	1.2	1.6	1.4	1.8	1.0	1.8	1.4	1.2
CLASSICAL	d017826	0.8	0.0	1.2	0.4	0.0	1.0	1.2	1.2	0.4	0.8	1.0	1.0
CLASSICAL	d019684	1.8	0.4	1.8	0.6	0.6	1.2	1.8	1.2	0.6	1.0	1.6	1.6
COUNTRY	b008293	1.0	0.0	1.2	1.0	0.6	0.8	1.2	0.8	0.6	1.2	1.2	1.0
COUNTRY	e003167	0.8	0.0	1.0	0.4	0.2	0.6	0.4	0.2	0.8	0.8	0.8	0.8
COUNTRY	e005922	0.2	0.4	2.0	1.4	1.8	1.4	1.6	1.2	1.0	1.8	0.6	0.6
COUNTRY	e008013	0.2	0.0	0.8	0.0	0.0	0.6	0.4	0.2	0.4	0.2	0.2	0.4
COUNTRY	e008637	1.6	0.0	1.6	1.8	0.4	1.8	2.0	2.0	1.2	2.0	2.0	1.6
COUNTRY	e009438	0.0	0.4	1.0	0.8	1.2	1.6	1.6	1.0	0.0	2.0	1.6	1.0
COUNTRY	e011578	0.8	0.4	0.8	0.6	1.0	1.2	1.2	1.0	0.0	1.2	1.0	1.0
COUNTRY	e011702	1.0	0.0	1.8	2.0	2.0	2.0	1.8	1.6	2.0	2.0	2.0	2.0
COUNTRY	e014903	1.8	0.0	1.6	1.4	1.6	1.2	1.8	0.4	1.2	1.6	1.4	1.6
COUNTRY	e018325	0.2	0.0	0.6	0.4	0.2	0.6	0.6	0.0	0.2	0.2	0.6	0.4
EDANCE	a002630	0.0	0.4	2.0	1.6	2.0	2.0	2.0	2.0	1.4	2.0	1.0	0.8
EDANCE	a003317	0.4	0.2	0.4	0.8	0.2	1.2	1.0	0.4	1.4	1.8	0.8	1.2
EDANCE	a003588	1.0	0.6	0.8	1.0	1.2	1.0	1.0	0.4	0.4	1.0	1.2	1.2
EDANCE	a006326	0.2	0.2	0.4	0.2	0.8	1.6	0.6	1.4	1.4	2.0	0.2	0.2
EDANCE	b007008	1.4	0.2	1.8	1.6	2.0	1.8	1.6	2.0	1.0	1.8	2.0	2.0
EDANCE	b008903	1.2	0.2	1.6	1.2	1.8	1.6	2.0	1.4	0.6	1.4	1.4	1.6
EDANCE	b009057	1.0	0.0	1.4	0.4	0.8	1.6	1.2	0.6	0.0	2.0	1.2	1.6
EDANCE	f000987	0.0	0.4	1.6	1.8	1.2	0.8	1.6	1.2	1.2	1.4	1.2	1.2
EDANCE	f005146	0.4	0.8	2.0	2.0	2.0	2.0	2.0	2.0	2.0	1.8	2.0	2.0
EDANCE	f010067	1.4	0.4	1.0	0.6	0.8	2.0	1.2	1.4	1.0	1.8	1.6	1.6
JAZZ	e001054	0.0	0.4	1.8	1.4	1.4	1.6	1.8	1.4	1.2	1.6	1.6	1.8
JAZZ	e004497	0.0	0.2	2.0	2.0	1.8	2.0	2.0	2.0	1.0	1.6	2.0	2.0
JAZZ	e005048	0.0	0.2	2.0	1.6	1.6	2.0	2.0	2.0	1.2	1.6	2.0	2.0
JAZZ	e007252	1.0	1.0	1.8	1.4	1.4	1.6	1.0	1.0	1.0	1.2	1.2	1.2
JAZZ	e007772	0.0	0.2	0.6	0.6	1.0	1.4	1.4	1.8	0.8	1.0	0.8	0.4
JAZZ	e007968	0.0	0.2	0.2	0.0	0.4	1.0	0.4	1.0	0.2	1.0	0.8	0.2
JAZZ	e009441	0.0	0.2	1.2	0.6	1.4	1.8	1.6	1.8	1.6	2.0	1.4	1.0
JAZZ	e010478	0.0	0.0	2.0	1.8	2.0	1.8	1.6	1.6	1.2	1.6	2.0	2.0
JAZZ	e016650	1.4	1.2	0.4	0.4	0.2	1.0	1.0	1.0	1.0	1.0	1.2	0.6
JAZZ	e019093	1.2	0.8	1.0	0.6	1.4	2.0	1.6	1.6	1.8	1.8	1.2	1.2
METAL	b001148	0.0	0.4	2.0	0.4	1.2	1.8	2.0	1.8	1.0	2.0	2.0	2.0
METAL	b002842	2.0	0.2	2.0	1.8	2.0	2.0	1.8	1.6	0.6	2.0	2.0	2.0
METAL	b005320	1.8	0.0	1.4	1.2	1.2	1.6	1.2	0.0	0.2	1.6	1.2	1.4
METAL	b005832	0.0	0.2	1.4	1.0	1.4	1.0	1.0	1.2	1.0	2.0	1.6	1.2
METAL	b006870	1.2	0.0	1.4	0.2	0.6	1.2	1.2	1.0	0.2	1.8	1.2	1.0
METAL	b010213	0.0	0.4	2.0	2.0	1.8	1.8	1.6	1.8	1.8	2.0	1.8	1.6
METAL	b010569	0.4	0.4	2.0	2.0	2.0	2.0	2.0	2.0	0.0	1.8	1.6	1.0
METAL	b011148	0.0	0.4	2.0	2.0	1.8	2.0	1.8	1.6	1.2	1.8	1.8	1.8
METAL	b011701	0.0	0.4	2.0	1.8	1.8	2.0	2.0	1.6	1.4	1.8	1.8	1.8
METAL	f012092	0.0	0.0	1.6	1.2	0.0	1.0	1.6	0.8	0.0	1.8	1.0	1.4
RAPHIPHOP	a000154	1.0	0.8	0.6	0.8	0.4	0.4	0.2	0.4	0.4	0.8	0.4	0.8
RAPHIPHOP	a000895	0.8	0.8	0.4	0.6	0.2	0.8	1.0	1.0	0.8	0.8	0.4	0.2
RAPHIPHOP	a001600	1.6	0.8	1.2	1.0	1.0	1.6	1.2	2.0	1.8	1.8	1.6	1.8
RAPHIPHOP	a003760	0.0	0.0	0.4	0.4	0.4	0.0	0.0	0.8	0.2	1.2	0.4	0.2
RAPHIPHOP	a004471	1.6	1.4	0.8	0.8	0.2	0.6	0.4	1.6	1.4	0.6	0.8	0.6
RAPHIPHOP	a006304	1.0	1.0	0.8	0.4	0.0	1.2	0.8	1.2	0.2	1.0	0.8	1.0
RAPHIPHOP	a008309	0.8	0.4	1.0	1.0	0.4	0.8	0.8	0.6	0.8	0.8	0.0	0.2
RAPHIPHOP	b000472	0.4	0.6	1.0	0.8	1.0	1.8	1.0	0.6	0.2	0.6	0.6	0.2
RAPHIPHOP	b001007	0.0	0.0	0.4	0.0	0.0	0.6	0.0	0.2	0.4	0.2	0.4	0.2
RAPHIPHOP	b011158	2.0	1.6	1.2	0.4	0.0	1.8	0.6	2.0	0.2	1.2	1.2	1.2
ROCKROLL	b000264	1.0	0.0	0.6	0.6	0.4	0.6	0.8	1.0	0.6	1.4	0.8	0.4
ROCKROLL	b000459	1.6	0.6	1.2	0.8	0.2	1.0	0.6	1.4	0.2	1.2	1.4	1.6
ROCKROLL	b001686	0.8	0.0	0.0	0.4	0.4	0.8	0.2	0.2	0.2	0.8	1.0	0.4
ROCKROLL	b005863	1.2	1.2	1.0	0.4	0.4	1.0	1.6	1.4	1.8	1.6	1.6	0.8
ROCKROLL	b008067	1.4	0.6	1.6	1.6	1.2	1.2	1.4	0.2	1.0	1.4	1.6	0.8
ROCKROLL	b009038	1.4	0.6	1.4	0.8	0.4	0.4	1.0	0.6	0.8	0.6	0.4	0.4
ROCKROLL	b012623	0.0	0.2	0.0	0.2	0.0	0.0	0.0	0.0	1.4	0.0	0.0	0.4
ROCKROLL	b015975	1.0	0.8	0.4	0.6	1.2	0.8	0.4	0.8	1.6	0.6	0.8	1.0
ROCKROLL	b016172	1.2	0.0	1.0	0.2	0.0	0.2	1.0	0.2	0.6	1.0	1.0	0.8
ROCKROLL	b019890	1.2	1.0	1.8	2.0	2.0	1.8	1.6	1.2	1.4	1.4	1.6	1.4
ROMANTIC	d000095	1.4	0.8	1.2	1.0	0.8	1.4	1.2	0.8	0.6	1.2	1.2	0.8
ROMANTIC	d000491	0.0	0.0	0.8	0.4	0.4	0.4	0.8	1.4	0.4	1.2	0.0	0.0
ROMANTIC	d003906	0.2	0.0	1.0	0.4	0.2	0.4	0.4	0.4	0.0	1.4	0.4	1.4
ROMANTIC	d005430	0.8	0.4	1.4	0.8	0.0	0.8	1.0	1.0	0.8	1.2	0.2	0.2
ROMANTIC	d008298	0.4	0.4	1.0	0.4	0.4	0.6	0.6	0.8	0.2	0.2	0.2	0.2
ROMANTIC	d008438	0.4	0.6	1.4	0.6	0.4	0.6	1.2	0.2	0.8	1.2	1.0	0.6
ROMANTIC	d009148	1.8	0.0	1.4	0.8	1.4	1.0	1.0	1.4	1.6	1.0	1.2	1.4
ROMANTIC	d009455	1.6	1.6	2.0	1.2	0.8	1.0	2.0	1.2	1.6	2.0	1.2	1.2
ROMANTIC	d010229	1.6	0.0	2.0	2.0	1.6	2.0	2.0	1.0	0.0	1.2	1.6	1.2
ROMANTIC	d010380	1.2	0.4	0.8	0.4	0.4	0.8	0.6	0.2	1.0	0.4	0.6	0.6

download these results as csv

Anonymized Metadata

Anonymized Metadata

Raw Scores

The raw data derived from the Evalutron 6000 human evaluations are located on the 2007:Audio Music Similarity and Retrieval Raw Data page.