2007:Query-by-Singing/Humming Results

Introduction

These are the results for the 2007 running of the Query-by-Singing/Humming task. For background information about this task set please refer to the 2007:Query by Singing/Humming page.

Task Descriptions

Task 1 Goto Task 1 Results: The first subtask is the same as last years. In this subtask, submitted systems take a sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth is calculated over the top 20 returns. The test database consists of 48 ground-truth MIDIs + 2000 Essen Collection MIDI noise files. See ESAC Data Homepage for more information about the Essen Collection. The query database consists of 2797 sung queries.

Task 2 Goto Task 2 Results: In the second subtask, the same setup as the first subtask used with combination of different transcribers and matchers. The test databases consists of 106 ground-truth MIDIS + 2000 Essen Collection MIDI noise files. The query databases consists of 355 sung queries.

General Legend

Team ID

FH = Pascal Ferraro, Pierre Hanna, Julien Allali, Matthias Robine
CG = Carlos G├│mez, Soraya Abad-Mota, Edna Ruckhaus
RJ1 = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1
RJ2 = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2
NM = Kjell Lemström, Niko Mikkilä
XW1 = Xiao Wu, Ming Li 1
XW2 = Xiao Wu, Ming Li 2
AU1 = Alexandra L. Uitdenbogerd 1
AU2 = Alexandra L. Uitdenbogerd 2
AU3 = Alexandra L. Uitdenbogerd 3

Task 1 Results

The first subtask is the same as last years. In this subtask, submitted systems take a sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth is calculated over the top 20 returns. The test database consists of 48 ground-truth MIDIs + 2000 Essen Collection MIDI noise files. The query database consists of 2797 sung queries.

Task 1 Overall Results

Teams	AU1	AU2	AU3	CG	NM	FH	RJ1	RJ2	XW1	XW2
Task 1 (MRR)	0.240	0.093	0.110	0.477	0.576	0.355	0.704	0.872	0.909	0.925

download these results as csv

Task 1 Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

Friedman's ANOVA Table
Source	SS	df	MS	Chi-sq	Prob>Chi-sq
Columns	3.5829e+003	9	398.0984	393.0438	0
Error	355.1146	423	0.8395
Total	3938	479

download these results as csv

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
AU1	AU2	-0.3039	1.6458	3.5956	FALSE
AU1	AU3	-0.4706	1.4792	3.4289	FALSE
AU1	CG	-4.1164	-2.1667	-0.2169	TRUE
AU1	NM	-4.6998	-2.75	-0.8002	TRUE
AU1	FH	-2.6373	-0.6875	1.2623	FALSE
AU1	RJ1	-5.5435	-3.5938	-1.644	TRUE
AU1	RJ2	-7.106	-5.1563	-3.2065	TRUE
AU1	XW1	-7.5539	-5.6042	-3.6544	TRUE
AU1	XW2	-8.2414	-6.2917	-4.3419	TRUE
AU2	AU3	-2.1164	-0.1667	1.7831	FALSE
AU2	CG	-5.7623	-3.8125	-1.8627	TRUE
AU2	NM	-6.3456	-4.3958	-2.4461	TRUE
AU2	FH	-4.2831	-2.3333	-0.3836	TRUE
AU2	RJ1	-7.1894	-5.2396	-3.2898	TRUE
AU2	RJ2	-8.7519	-6.8021	-4.8523	TRUE
AU2	XW1	-9.1998	-7.25	-5.3002	TRUE
AU2	XW2	-9.8873	-7.9375	-5.9877	TRUE
AU3	CG	-5.5956	-3.6458	-1.6961	TRUE
AU3	NM	-6.1789	-4.2292	-2.2794	TRUE
AU3	FH	-4.1164	-2.1667	-0.2169	TRUE
AU3	RJ1	-7.0227	-5.0729	-3.1231	TRUE
AU3	RJ2	-8.5852	-6.6354	-4.6856	TRUE
AU3	XW1	-9.0331	-7.0833	-5.1336	TRUE
AU3	XW2	-9.7206	-7.7708	-5.8211	TRUE
CG	NM	-2.5331	-0.5833	1.3664	FALSE
CG	FH	-0.4706	1.4792	3.4289	FALSE
CG	RJ1	-3.3769	-1.4271	0.5227	FALSE
CG	RJ2	-4.9394	-2.9896	-1.0398	TRUE
CG	XW1	-5.3873	-3.4375	-1.4877	TRUE
CG	XW2	-6.0748	-4.125	-2.1752	TRUE
NM	FH	0.1127	2.0625	4.0123	TRUE
NM	RJ1	-2.7935	-0.8438	1.106	FALSE
NM	RJ2	-4.356	-2.4063	-0.4565	TRUE
NM	XW1	-4.8039	-2.8542	-0.9044	TRUE
NM	XW2	-5.4914	-3.5417	-1.5919	TRUE
FH	RJ1	-4.856	-2.9063	-0.9565	TRUE
FH	RJ2	-6.4185	-4.4688	-2.519	TRUE
FH	XW1	-6.8664	-4.9167	-2.9669	TRUE
FH	XW2	-7.5539	-5.6042	-3.6544	TRUE
RJ1	RJ2	-3.5123	-1.5625	0.3873	FALSE
RJ1	XW1	-3.9602	-2.0104	-0.0606	TRUE
RJ1	XW2	-4.6477	-2.6979	-0.7481	TRUE
RJ2	XW1	-2.3977	-0.4479	1.5019	FALSE
RJ2	XW2	-3.0852	-1.1354	0.8144	FALSE
XW1	XW2	-2.6373	-0.6875	1.2623	FALSE

download these results as csv File:2007 Qbsh07 task1 friedmans.png

Task 1 Summary Results by Query Group

Group ID	Count	AU1	AU2	AU3	CG	NM	FH	RJ1	RJ2	XW1	XW2
1	9	0.023	0.010	0.006	0.423	0.726	0.347	0.722	0.944	1.000	1.000
2	9	0.059	0.120	0.117	0.598	0.578	0.056	0.778	0.806	0.444	0.455
3	8	0.500	0.250	0.188	1.000	0.700	0.438	0.875	0.875	0.875	0.875
4	9	0.519	0.444	0.472	0.786	0.901	0.467	1.000	1.000	0.944	1.000
5	9	0.333	0.222	0.229	0.309	0.494	0.278	0.083	0.561	0.917	1.000
6	8	0.188	0.000	0.031	0.380	0.563	0.080	0.760	1.000	1.000	1.000
7	9	0.236	0.111	0.111	0.346	0.722	0.287	0.889	0.944	0.897	0.944
8	9	0.160	0.111	0.111	0.401	0.615	0.228	0.800	0.562	1.000	1.000
9	8	0.196	0.375	0.375	0.688	0.608	0.138	0.503	1.000	1.000	1.000
10	10	0.426	0.064	0.032	0.684	0.810	0.660	1.000	0.825	1.000	1.000
11	62	0.349	0.142	0.194	0.541	0.627	0.519	0.685	0.922	0.922	0.929
12	99	0.352	0.119	0.212	0.684	0.701	0.498	0.838	0.926	0.932	0.934
13	105	0.111	0.068	0.094	0.302	0.640	0.343	0.810	0.922	0.946	0.956
14	99	0.378	0.170	0.235	0.643	0.661	0.347	0.740	0.907	0.923	0.940
15	15	0.157	0.101	0.185	0.401	0.561	0.229	0.636	0.794	0.833	0.826
16	101	0.493	0.173	0.226	0.529	0.683	0.568	0.703	0.874	0.924	0.940
17	95	0.099	0.060	0.045	0.224	0.425	0.065	0.410	0.787	0.942	0.961
18	104	0.244	0.173	0.211	0.562	0.657	0.362	0.709	0.913	0.988	0.995
19	104	0.328	0.107	0.131	0.653	0.651	0.440	0.876	0.956	0.938	0.954
20	108	0.140	0.062	0.133	0.318	0.457	0.210	0.753	0.943	0.959	0.972
21	46	0.182	0.022	0.018	0.438	0.433	0.263	0.487	0.649	0.655	0.665
22	108	0.232	0.104	0.160	0.560	0.797	0.407	0.850	0.916	0.987	0.986
23	45	0.394	0.044	0.000	0.676	0.480	0.478	0.703	0.961	0.944	0.956
24	75	0.142	0.029	0.062	0.146	0.347	0.135	0.645	0.821	0.773	0.785
25	77	0.169	0.019	0.015	0.392	0.553	0.318	0.205	0.499	0.685	0.708
26	39	0.372	0.088	0.024	0.487	0.533	0.309	0.768	0.893	0.885	0.897
27	52	0.206	0.019	0.020	0.705	0.429	0.395	0.808	0.971	0.971	0.971
28	55	0.149	0.011	0.010	0.189	0.067	0.225	0.352	0.519	0.809	0.839
29	97	0.195	0.071	0.075	0.256	0.378	0.200	0.824	0.860	0.944	0.958
30	87	0.331	0.187	0.216	0.401	0.640	0.374	0.802	0.933	0.908	0.943
31	90	0.217	0.135	0.143	0.677	0.651	0.387	0.899	0.942	0.969	1.000
32	85	0.246	0.058	0.015	0.545	0.702	0.499	0.431	0.891	0.951	0.975
33	76	0.135	0.018	0.007	0.399	0.423	0.318	0.526	0.781	0.653	0.722
34	89	0.181	0.044	0.041	0.272	0.529	0.393	0.633	0.856	0.900	0.934
35	93	0.111	0.057	0.056	0.340	0.447	0.234	0.715	0.946	0.893	0.916
36	25	0.354	0.141	0.214	0.377	0.811	0.415	0.920	0.920	0.920	0.960
37	39	0.237	0.139	0.130	0.575	0.452	0.431	0.600	0.811	0.932	0.949
38	68	0.189	0.102	0.014	0.756	0.492	0.450	0.883	0.891	0.925	0.912
39	95	0.318	0.088	0.133	0.610	0.677	0.400	0.773	0.864	0.906	0.891
40	52	0.181	0.067	0.103	0.320	0.461	0.292	0.716	0.837	0.877	0.904
41	77	0.209	0.109	0.135	0.310	0.702	0.434	0.694	0.882	0.961	0.956
42	60	0.391	0.133	0.138	0.626	0.685	0.433	0.810	0.929	0.919	0.953
43	73	0.276	0.131	0.172	0.621	0.676	0.386	0.565	0.923	0.948	0.970
44	48	0.340	0.062	0.035	0.567	0.720	0.493	0.739	0.928	0.958	0.958
45	65	0.088	0.066	0.066	0.425	0.662	0.283	0.716	0.878	0.901	0.928
46	49	0.226	0.020	0.005	0.550	0.545	0.459	0.757	0.893	0.949	0.963
47	20	0.115	0.000	0.013	0.491	0.663	0.224	0.821	0.925	0.967	0.967
48	32	0.254	0.110	0.096	0.586	0.394	0.243	0.868	0.880	0.956	0.940

download these results as csv

Task 2 Results

In this subtask, the same setup as the first subtask used with combination of different transcribers and matchers. The test databases consists of 106 ground-truth MIDIS + 2000 Essen Collection MIDI noise files. The query databases consists of 355 sung queries.

Task 2 Legend

Team ID

FH_XW = Pascal Ferraro, Pierre Hanna, Julien Allali, Matthias Robine based on XW note transcriber
CG_XW = Carlos G├│mez, Soraya Abad-Mota, Edna Ruckhaus based on XW note transcriber
RJ1_RJ = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1 based on RJ pitch transcriber
RJ1_XW = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1 based on XW pitch transcriber
RJ2_RJ = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2 based on RJ pitch transcriber
RJ2_XW = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2 based on XW pitch transcriber
NM_XW = Kjell Lemström, Niko Mikkilä based on XW note transcriber
XW1_XW = Xiao Wu, Ming Li 1 based on XW note transcriber
XW2_XW = Xiao Wu, Ming Li 2 based on XW pitch transcriber

Task 2 Overall Results

Combinations	CG_XW	FH_XW	NM_XW	RJ1_RJ	RJ1_XW	RJ2_RJ	RJ2_XW	XW1_XW	XW2_XW	XW2_RJ
Task II (MRR)	0.715	0.452	0.618	0.345	0.305	0.536	0.379	0.917	0.937	0.883

download these results as csv

Task 2 Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

Friedman's ANOVA Table
Source	SS	df	MS	Chi-sq	Prob>Chi-sq
Columns	3.4149e+003	9	379.4329	451.1891	0
Error	3.8056e+003	945	4.0271
Total	7.2205e+003	1059

download these results as csv

ComboID	ComboID	Lowerbound	Mean	Upperbound	Significance
CG_XW	NM_XW	-0.3748	0.8208	2.0163	FALSE
CG_XW	FH_XW	0.7054	1.9009	3.0965	TRUE
CG_XW	RJ1_RJ	1.6582	2.8538	4.0493	TRUE
CG_XW	RJ1_XW	1.9365	3.1321	4.3276	TRUE
CG_XW	RJ2_RJ	0.4696	1.6651	2.8606	TRUE
CG_XW	RJ2_XW	1.3469	2.5425	3.738	TRUE
CG_XW	XW1_XW	-2.7333	-1.5377	-0.3422	TRUE
CG_XW	XW2_XW	-2.988	-1.7925	-0.5969	TRUE
CG_XW	XW2_RJ	-2.5729	-1.3774	-0.1818	TRUE
NM_XW	FH_XW	-0.1154	1.0802	2.2757	FALSE
NM_XW	RJ1_RJ	0.8375	2.033	3.2286	TRUE
NM_XW	RJ1_XW	1.1158	2.3113	3.5069	TRUE
NM_XW	RJ2_RJ	-0.3512	0.8443	2.0399	FALSE
NM_XW	RJ2_XW	0.5262	1.7217	2.9172	TRUE
NM_XW	XW1_XW	-3.554	-2.3585	-1.1629	TRUE
NM_XW	XW2_XW	-3.8087	-2.6132	-1.4177	TRUE
NM_XW	XW2_RJ	-3.3937	-2.1981	-1.0026	TRUE
FH_XW	RJ1_RJ	-0.2427	0.9528	2.1484	FALSE
FH_XW	RJ1_XW	0.0356	1.2311	2.4267	TRUE
FH_XW	RJ2_RJ	-1.4314	-0.2358	0.9597	FALSE
FH_XW	RJ2_XW	-0.554	0.6415	1.8371	FALSE
FH_XW	XW1_XW	-4.6342	-3.4387	-2.2431	TRUE
FH_XW	XW2_XW	-4.8889	-3.6934	-2.4979	TRUE
FH_XW	XW2_RJ	-4.4738	-3.2783	-2.0828	TRUE
RJ1_RJ	RJ1_XW	-0.9172	0.2783	1.4738	FALSE
RJ1_RJ	RJ2_RJ	-2.3842	-1.1887	0.0069	FALSE
RJ1_RJ	RJ2_XW	-1.5069	-0.3113	0.8842	FALSE
RJ1_RJ	XW1_XW	-5.5871	-4.3915	-3.196	TRUE
RJ1_RJ	XW2_XW	-5.8418	-4.6462	-3.4507	TRUE
RJ1_RJ	XW2_RJ	-5.4267	-4.2311	-3.0356	TRUE
RJ1_XW	RJ2_RJ	-2.6625	-1.467	-0.2714	TRUE
RJ1_XW	RJ2_XW	-1.7852	-0.5896	0.6059	FALSE
RJ1_XW	XW1_XW	-5.8654	-4.6698	-3.4743	TRUE
RJ1_XW	XW2_XW	-6.1201	-4.9245	-3.729	TRUE
RJ1_XW	XW2_RJ	-5.705	-4.5094	-3.3139	TRUE
RJ2_RJ	RJ2_XW	-0.3182	0.8774	2.0729	FALSE
RJ2_RJ	XW1_XW	-4.3984	-3.2028	-2.0073	TRUE
RJ2_RJ	XW2_XW	-4.6531	-3.4575	-2.262	TRUE
RJ2_RJ	XW2_RJ	-4.238	-3.0425	-1.8469	TRUE
RJ2_XW	XW1_XW	-5.2757	-4.0802	-2.8846	TRUE
RJ2_XW	XW2_XW	-5.5304	-4.3349	-3.1394	TRUE
RJ2_XW	XW2_RJ	-5.1154	-3.9198	-2.7243	TRUE
XW1_XW	XW2_XW	-1.4503	-0.2547	0.9408	FALSE
XW1_XW	XW2_RJ	-1.0352	0.1604	1.3559	FALSE
XW2_XW	XW2_RJ	-0.7804	0.4151	1.6106	FALSE

download these results as csv File:2007 Qbsh07 task2 friedmans.png

Task 2 Summary Results by Query Group

Group ID	Count	CG_XW	NM_XW	FH_XW	RJ1_RJ	RJ1_XW	RJ2_RJ	RJ2_XW	XW1_XW	XW2_XW	XW2_RJ
24	11	0.735	0.695	0.424	0.318	0.364	0.783	0.263	0.927	0.955	0.864
25	5	0.814	0.667	0.180	0.600	0.295	0.667	0.149	1.000	1.000	1.000
65	1	0.167	0.000	0.000	0.000	0.000	0.000	0.000	0.111	0.250	0.143
75	2	0.545	0.000	0.250	0.000	0.000	0.000	0.000	1.000	1.000	1.000
80	1	0.000	0.000	0.143	0.000	0.000	0.000	0.000	0.000	0.000	1.000
83	6	0.639	0.667	0.678	0.000	0.000	0.667	0.667	0.682	0.690	0.750
89	6	0.861	1.000	0.700	1.000	1.000	1.000	1.000	1.000	1.000	1.000
101	3	1.000	0.030	0.000	0.056	0.048	0.017	0.000	1.000	1.000	1.000
106	5	0.733	0.411	0.200	0.000	0.000	0.233	0.050	1.000	1.000	1.000
108	2	0.750	1.000	0.000	0.000	0.000	0.031	0.071	1.000	1.000	1.000
110	2	0.417	0.063	0.029	0.000	0.000	0.050	0.083	0.625	0.750	1.000
111	1	1.000	1.000	0.500	1.000	1.000	0.000	0.000	1.000	1.000	1.000
114	5	0.458	0.450	0.022	0.000	0.000	0.400	0.300	1.000	1.000	0.800
116	2	0.600	0.500	0.250	0.000	0.000	0.500	0.125	1.000	1.000	1.000
120	2	0.500	0.000	0.000	0.000	0.000	0.500	0.528	0.600	1.000	0.750
128	4	0.400	0.642	0.516	0.156	0.000	0.053	0.287	1.000	1.000	1.000
129	1	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.333
133	2	0.500	1.000	0.500	0.000	0.500	0.600	1.000	1.000	1.000	1.000
139	2	1.000	0.667	0.313	0.000	0.000	1.000	0.625	1.000	1.000	1.000
140	2	0.375	0.042	0.000	0.000	0.000	0.000	0.000	0.333	0.417	0.500
160	2	0.750	0.750	0.500	0.061	0.036	0.250	0.125	0.525	0.550	0.500
162	9	0.790	0.674	0.365	0.444	0.278	0.911	0.590	0.944	0.917	0.787
167	1	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000
185	2	1.000	1.000	0.750	0.000	0.000	1.000	0.500	1.000	1.000	1.000
192	2	1.000	1.000	0.533	0.083	0.100	0.500	0.500	1.000	1.000	0.500
193	11	0.740	0.778	0.327	0.365	0.286	0.360	0.291	1.000	1.000	1.000
195	4	0.750	1.000	0.750	0.250	0.250	0.453	0.750	1.000	1.000	1.000
197	4	0.625	0.500	0.125	0.500	0.161	0.750	0.500	1.000	1.000	1.000
210	1	1.000	0.000	0.000	0.250	0.250	0.000	0.000	1.000	1.000	1.000
232	4	1.000	0.633	0.750	1.000	1.000	1.000	1.000	1.000	1.000	1.000
233	1	1.000	1.000	0.000	1.000	1.000	1.000	0.167	1.000	1.000	1.000
245	7	0.714	0.679	0.571	0.000	0.000	0.786	0.378	1.000	1.000	1.000
250	5	0.700	1.000	1.000	0.000	0.000	0.800	0.511	0.662	0.718	0.414
265	2	0.500	0.000	0.571	0.000	0.000	0.500	0.083	0.500	0.500	0.500
271	1	1.000	0.000	0.000	0.000	0.000	0.000	0.000	1.000	1.000	1.000
276	5	1.000	0.247	0.200	0.000	0.000	0.425	0.350	0.800	0.867	0.800
278	1	0.000	0.000	0.091	0.000	0.000	1.000	0.000	0.000	0.063	0.000
279	2	1.000	0.500	0.500	0.500	0.500	0.500	0.500	1.000	1.000	1.000
289	7	0.524	0.310	0.171	0.429	0.333	0.722	0.361	1.000	1.000	1.000
293	2	0.750	1.000	1.000	1.000	1.000	1.000	0.625	1.000	1.000	0.000
306	8	0.604	0.813	0.639	0.250	0.250	0.427	0.046	0.875	0.875	0.875
319	12	0.651	0.521	0.613	0.625	0.508	0.325	0.121	0.875	1.000	0.917
321	9	0.411	0.376	0.175	0.593	0.358	0.458	0.186	0.763	0.833	0.903
362	9	0.750	0.704	0.306	0.556	0.398	0.926	0.648	1.000	1.000	1.000
363	3	0.667	0.381	0.000	1.000	0.833	0.667	0.667	1.000	1.000	1.000
392	4	0.583	0.250	0.250	0.031	0.000	0.268	0.250	1.000	1.000	0.750
399	4	0.781	0.375	0.396	0.313	0.119	0.467	0.018	1.000	1.000	1.000
438	1	1.000	1.000	1.000	1.000	0.071	0.000	0.067	1.000	1.000	1.000
461	1	1.000	1.000	0.000	1.000	1.000	1.000	0.333	1.000	1.000	1.000
468	1	1.000	1.000	0.125	1.000	1.000	1.000	1.000	1.000	1.000	1.000
475	4	0.583	0.750	1.000	0.250	0.125	0.625	0.375	1.000	1.000	1.000
513	4	0.411	0.088	0.250	0.125	0.000	0.290	0.333	1.000	1.000	1.000
514	6	0.679	0.649	0.556	0.333	0.192	1.000	0.408	1.000	1.000	1.000
515	6	0.639	0.667	0.167	0.308	0.431	0.392	0.722	1.000	1.000	1.000
529	4	0.769	0.750	0.500	0.000	0.000	0.000	0.013	1.000	1.000	1.000
533	2	0.500	0.500	0.500	0.500	0.500	0.500	0.500	0.500	0.500	0.000
540	5	0.712	1.000	0.707	0.767	1.000	0.667	0.850	1.000	1.000	0.500
550	1	1.000	1.000	0.083	0.000	0.000	0.067	0.167	1.000	1.000	1.000
561	3	0.511	1.000	0.733	0.000	0.000	1.000	0.778	1.000	1.000	1.000
562	2	0.417	0.110	0.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000
568	1	1.000	1.000	1.000	0.000	0.000	0.167	0.500	1.000	1.000	1.000
570	3	0.833	0.667	0.667	0.037	0.067	0.444	0.377	1.000	1.000	1.000
572	2	0.600	0.500	1.000	1.000	0.750	0.750	0.500	1.000	1.000	1.000
573	1	0.125	1.000	1.000	0.000	0.000	0.333	0.250	1.000	1.000	1.000
594	3	1.000	0.500	0.000	0.000	0.000	0.364	0.208	1.000	1.000	1.000
606	4	1.000	0.769	0.750	0.333	0.500	0.542	0.750	1.000	1.000	0.750
628	3	0.667	0.833	0.394	0.000	0.000	1.000	0.167	1.000	1.000	1.000
636	2	0.250	1.000	0.025	1.000	0.288	0.550	0.000	0.667	0.667	1.000
643	4	0.875	0.750	0.125	0.050	0.013	0.500	0.300	1.000	1.000	1.000
647	4	1.000	0.786	0.641	0.875	0.833	0.606	0.750	1.000	1.000	1.000
660	3	0.500	1.000	0.833	0.000	0.000	0.370	0.167	0.421	0.417	0.667
662	3	0.833	0.367	0.667	1.000	1.000	0.567	0.567	1.000	1.000	1.000
685	4	0.750	1.000	1.000	1.000	0.875	0.750	0.354	1.000	1.000	1.000
693	2	1.000	1.000	1.000	1.000	1.000	0.045	0.167	1.000	1.000	1.000
728	3	0.778	0.333	0.208	0.333	0.333	0.333	0.361	1.000	1.000	1.000
730	2	1.000	0.500	0.276	0.000	0.000	0.417	0.050	0.550	1.000	0.500
732	10	0.870	0.900	1.000	1.000	0.950	0.950	0.491	1.000	1.000	1.000
736	1	1.000	0.500	1.000	0.056	1.000	0.111	0.500	1.000	1.000	1.000
747	2	1.000	1.000	0.500	0.500	0.500	1.000	1.000	1.000	1.000	1.000
770	3	1.000	1.000	0.750	0.333	0.333	0.750	0.444	1.000	1.000	1.000
790	6	1.000	0.769	0.690	0.250	0.333	0.696	0.271	0.917	1.000	0.667
794	2	1.000	1.000	1.000	0.000	0.000	1.000	1.000	1.000	1.000	1.000
805	3	0.333	0.333	0.186	0.000	0.000	0.148	0.111	1.000	1.000	0.667
811	4	0.377	0.000	0.046	0.000	0.000	0.000	0.000	0.750	0.750	0.000
839	4	0.750	1.000	0.181	0.500	0.500	1.000	1.000	1.000	1.000	1.000
850	1	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000
858	3	1.000	1.000	1.000	0.000	0.000	1.000	0.483	1.000	1.000	1.000
914	1	1.000	1.000	0.333	0.000	0.000	0.000	0.000	1.000	1.000	1.000
915	7	0.929	0.729	0.786	0.143	0.143	0.143	0.286	0.857	0.857	0.905
926	2	1.000	0.500	0.500	0.000	0.000	0.000	0.026	1.000	1.000	1.000
937	2	0.600	0.500	0.083	0.000	0.000	0.250	0.500	1.000	1.000	1.000
941	2	0.500	0.538	1.000	0.500	0.500	0.167	0.071	1.000	1.000	1.000
956	2	1.000	0.071	0.250	0.031	0.100	0.250	0.050	1.000	1.000	1.000
1000	1	0.500	0.500	1.000	0.000	0.000	0.000	1.000	1.000	1.000	1.000
1014	1	1.000	0.500	1.000	0.000	0.000	1.000	1.000	1.000	1.000	1.000
1036	1	0.000	0.000	0.000	0.056	0.071	0.000	0.000	1.000	1.000	1.000
1039	1	1.000	1.000	1.000	0.000	0.000	0.077	0.111	1.000	1.000	1.000
1044	1	1.000	1.000	0.333	0.000	0.000	0.167	0.000	1.000	1.000	1.000
1054	4	0.653	0.271	0.313	0.583	0.750	0.000	0.063	0.875	1.000	1.000
1062	3	1.000	0.750	0.356	0.333	0.333	0.108	0.111	1.000	1.000	1.000
1067	1	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
1068	1	0.250	1.000	1.000	0.000	0.000	0.000	0.000	1.000	1.000	1.000
1070	8	0.592	0.400	0.094	0.375	0.229	0.700	0.455	1.000	1.000	0.750
1082	3	0.833	0.300	0.083	0.000	0.000	0.733	0.667	1.000	1.000	1.000
1083	1	0.200	0.143	0.000	1.000	0.500	0.000	0.000	0.053	0.143	0.500
1094	3	0.500	0.061	0.000	0.000	0.000	0.047	0.048	0.694	1.000	1.000

download these results as csv