2011:Query-by-Singing/Humming Results

Introduction

These are the results for the 2008 running of the Query-by-Singing/Humming task. For background information about this task set please refer to the 2011:Query by Singing/Humming page.

Task Descriptions

Task 1 Goto Task 1 Results: The first subtask is the same as last year. In this subtask, submitted systems take a sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth, as well as the simple hit(1)/miss(0) counting, is calculated over the top 10 returns. Two data sets are used:

• Jang's Dataset Results Roger Jang's MIR-QBSH corpus with 48 songs as ground truth + 2000 Essen Collection MIDI noise files. See ESAC Data Homepage for more information about the Essen Collection. The queries consists of 4431 humming. All queries are from the beginning of references
• ThinkIT's Dataset Results IOACAS corpus 1 data set with 106 songs as ground truth + 2000 Essen Collection MIDI noise files. See ESAC Data Homepage for more information about the Essen Collection. The queries consists of 355 humming. There are no "singing from beginning" gurantee.
• IOACAS's 2nd Dataset Results IOACAS corpus 2 data set with 192 songs as ground truth + 2000 Essen Collection MIDI noise files. See ESAC Data Homepage for more information about the Essen Collection. The queries consists of 404 humming. There are no "singing from beginning" gurantee.

Task 2 Goto Task 2 Results: The second subtask is the query against other humming. In the second subtask, Roger Jang's MIR-QBSH corpus has been divided into two groups (2040 as queries and 2391 as database). The query is performed against the other humming database and the top 10 closed are returned. The score is simple count of how many returns belong to the same ground truth song.

General Legend

Sub code	Submission name	Abstract	Contributors
TY1	QBSH algorithm 1	PDF	Tzu-Chun Yeh
TY2	QBSH algorithm 2	PDF	Tzu-Chun Yeh
JSSLP1	KETI	PDF	Dalwon Jang, SUNG JOO PARK,Chai-Jong Song,Saim Shin,SEI-JIN JANG,Jong-Seol Lee,Seok-Pil Lee,Kyeung Hak Seo

Task 1 Results

Task 1a, Jang's dataset Results

Task 1a Overall Results

	JSSLP1	TY1	TY2
Simple Count	0.939	0.956	0.919
MRR	0.897	0.93	0.881
Total Count	4431	4431	4431

download these results as csv

Task 1a Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 1a Simple/MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

Simple Hit/Miss Count:

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
TY1	JSSLP1	-0.0749	0.3438	0.7624	FALSE
TY1	TY2	0.6439	1.0625	1.4811	TRUE
JSSLP1	TY2	0.3001	0.7188	1.1374	TRUE

download these results as csv

MRR Method:

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
TY1	JSSLP1	0.2321	0.6979	1.1637	TRUE
TY1	TY2	0.7426	1.2083	1.6741	TRUE
JSSLP1	TY2	0.0446	0.5104	0.9762	TRUE

download these results as csv

Task 1a Summary Results by Query Group

Simple Hit/Miss Count

	JSSLP1	TY1	TY2
1	0.947	0.947	0.895
2	0.714	0.786	0.643
3	0.727	0.727	0.727
4	0.8	0.8	0.8
5	0.933	0.933	0.8
6	0.833	0.833	0.917
7	0.867	0.933	0.867
8	0.933	0.933	0.933
9	0.917	0.917	0.917
10	0.947	0.684	0.842
11	0.927	0.948	0.896
12	0.968	0.968	0.961
13	0.95	0.981	0.963
14	0.975	0.994	0.956
15	0.963	0.963	0.963
16	0.931	0.969	0.925
17	0.941	0.98	0.928
18	0.97	0.976	0.957
19	0.949	0.968	0.943
20	0.971	1	0.982
21	0.742	0.818	0.758
22	0.977	0.988	0.977
23	1	0.984	0.952
24	0.752	0.843	0.785
25	0.756	0.862	0.715
26	0.962	0.962	0.962
27	0.965	0.965	0.965
28	0.879	0.879	0.725
29	0.993	0.987	0.973
30	0.959	0.986	0.938
31	0.979	0.993	0.972
32	0.964	0.956	0.934
33	0.94	0.94	0.862
34	0.949	0.927	0.891
35	0.926	0.966	0.886
36	0.953	0.93	0.953
37	0.939	0.939	0.848
38	0.965	0.93	0.956
39	0.952	0.98	0.939
40	0.93	0.972	0.915
41	0.944	0.984	0.913
42	0.957	0.946	0.946
43	0.951	0.976	0.943
44	0.95	0.95	0.95
45	0.97	0.97	0.939
46	0.989	0.978	0.935
47	0.931	1	0.931
48	0.962	1	0.925

download these results as csv

MRR Method

	JSSLP1	TY1	TY2
1	0.947	0.901	0.847
2	0.714	0.732	0.595
3	0.682	0.682	0.645
4	0.8	0.8	0.8
5	0.933	0.85	0.767
6	0.833	0.833	0.808
7	0.867	0.889	0.867
8	0.933	0.88	0.933
9	0.917	0.917	0.844
10	0.947	0.61	0.842
11	0.876	0.932	0.868
12	0.945	0.958	0.937
13	0.917	0.94	0.938
14	0.865	0.969	0.879
15	0.944	0.907	0.93
16	0.889	0.922	0.891
17	0.905	0.95	0.857
18	0.959	0.965	0.947
19	0.931	0.968	0.925
20	0.929	0.977	0.944
21	0.632	0.722	0.678
22	0.973	0.984	0.966
23	0.965	0.962	0.939
24	0.688	0.78	0.694
25	0.541	0.79	0.595
26	0.942	0.946	0.939
27	0.928	0.965	0.951
28	0.776	0.798	0.597
29	0.953	0.968	0.959
30	0.934	0.969	0.91
31	0.965	0.988	0.944
32	0.93	0.956	0.891
33	0.886	0.92	0.802
34	0.882	0.908	0.85
35	0.887	0.948	0.876
36	0.935	0.93	0.953
37	0.908	0.903	0.833
38	0.949	0.906	0.923
39	0.921	0.946	0.882
40	0.878	0.914	0.866
41	0.922	0.948	0.898
42	0.895	0.935	0.91
43	0.925	0.943	0.904
44	0.923	0.944	0.938
45	0.962	0.954	0.916
46	0.913	0.948	0.92
47	0.931	0.955	0.931
48	0.945	0.95	0.915

download these results as csv

Task 1a Summary Results by Query

Simple Hit/Miss Counting [1]

MRR Method [2]

Task 1b, ThinkIT's dataset Results

Task 1b Overall Results

	JSSLP1	TY1	TY2
Simple Count	0.941	0.456	0.899
MRR	0.912	0.437	0.853
Total Count	355	355	355

download these results as csv

Task 1b Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 1b Simple/MRR data over the 106 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

Simple Hit/Miss Count:

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
JSSLP1	TY2	-0.0922	0.1604	0.4130	FALSE
JSSLP1	TY1	0.8465	1.0991	1.3516	TRUE
TY2	TY1	0.6861	0.9387	1.1913	TRUE

download these results as csv

MRR Method:

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
JSSLP1	TY2	0.0002	0.2642	0.5281	TRUE
JSSLP1	TY1	0.8729	1.1368	1.4007	TRUE
TY2	TY1	0.6087	0.8726	1.1366	TRUE

download these results as csv

Task 1b Summary Results by Query Group

Simple Hit/Miss Count

	JSSLP1	TY1	TY2
1	1	0.636	1
2	1	0.6	0.8
3	0	0	1
4	1	1	1
5	0	0	1
6	0.833	0	0.667
7	1	1	1
8	1	0.333	0.667
9	1	0	1
10	1	0	1
11	1	0	0.5
12	1	1	1
13	0.8	0.2	0.6
14	1	0	1
15	1	0	0.5
16	1	1	0.75
17	0	0	0
18	1	0.5	1
19	1	0	1
20	1	0	0.5
21	1	0	0.5
22	1	0.444	0.889
23	1	1	1
24	1	0	1
25	1	0.5	1
26	1	0.636	0.909
27	1	0.5	1
28	1	0.5	0.75
29	1	1	1
30	1	1	1
31	1	1	1
32	1	0	1
33	1	0	1
34	0.5	0.5	0.5
35	1	0	1
36	0.8	0	0.8
37	1	0	0
38	1	0.5	1
39	1	0.429	1
40	1	1	1
41	0.875	0.25	0.875
42	1	0.667	0.917
43	0.889	0.667	0.778
44	1	0.556	1
45	1	1	1
46	0.75	0.5	0.75
47	1	0.75	1
48	1	1	1
49	1	1	1
50	1	1	1
51	1	0.5	1
52	1	0.5	0.75
53	1	0.333	1
54	1	0.833	1
55	1	0	1
56	0.5	0.5	0.5
57	1	1	1
58	1	0	1
59	1	0	1
60	0	1	1
61	1	0	1
62	1	0.333	1
63	1	1	1
64	1	0	1
65	1	0	0.667
66	1	0.5	0.75
67	0.333	0	1
68	1	1	1
69	1	0	1
70	1	1	1
71	1	0	1
72	1	1	1
73	1	1	1
74	1	1	1
75	1	0.333	0.667
76	0.5	0	1
77	1	1	1
78	1	1	1
79	1	0.5	1
80	0.333	0.333	1
81	1	0.333	1
82	1	0	1
83	1	0	1
84	0.75	0.25	0.5
85	1	0.5	1
86	1	1	1
87	1	0	1
88	1	0	1
89	1	0.571	0.714
90	1	0	1
91	1	0	1
92	1	0.5	1
93	1	0.5	1
94	1	0	1
95	1	0	1
96	1	1	0
97	1	0	1
98	1	0	1
99	1	0.75	1
100	1	0.333	1
101	0	0	0
102	1	0	1
103	1	0.375	1
104	1	0	1
105	1	1	0
106	0.667	0	0.667

download these results as csv

MRR Method

	JSSLP1	TY1	TY2
1	1	0.5	1
2	1	0.6	0.8
3	0	0	1
4	0.667	0.163	0.563
5	0	0	0.25
6	0.75	0	0.667
7	1	1	0.917
8	1	0.333	0.667
9	1	0	0.9
10	1	0	1
11	1	0	0.5
12	1	1	1
13	0.8	0.2	0.6
14	1	0	0.625
15	0.667	0	0.5
16	1	1	0.542
17	0	0	0
18	1	0.5	1
19	1	0	1
20	0.375	0	0.1
21	1	0	0.083
22	0.926	0.389	0.806
23	1	1	1
24	1	0	1
25	1	0.5	1
26	1	0.636	0.909
27	1	0.5	1
28	1	0.375	0.75
29	1	1	1
30	1	1	1
31	1	1	1
32	1	0	1
33	0.9	0	1
34	0.5	0.5	0.5
35	1	0	1
36	0.8	0	0.8
37	0.333	0	0
38	1	0.5	1
39	1	0.429	1
40	1	1	1
41	0.875	0.167	0.875
42	1	0.667	0.917
43	0.833	0.667	0.722
44	1	0.556	1
45	1	1	1
46	0.625	0.5	0.75
47	1	0.75	1
48	1	1	1
49	1	1	1
50	1	1	1
51	1	0.5	0.75
52	1	0.281	0.75
53	1	0.333	0.861
54	1	0.75	1
55	1	0	1
56	0.5	0.5	0.5
57	1	1	1
58	1	0	1
59	1	0	1
60	0	1	1
61	1	0	0.25
62	1	0.333	1
63	1	1	1
64	1	0	1
65	1	0	0.667
66	1	0.5	0.75
67	0.333	0	1
68	0.625	1	0.556
69	1	0	1
70	1	1	1
71	1	0	0.722
72	1	1	1
73	1	1	1
74	1	1	1
75	1	0.333	0.444
76	0.5	0	1
77	1	1	1
78	1	1	1
79	1	0.5	1
80	0.333	0.333	1
81	1	0.333	0.857
82	1	0	1
83	0.833	0	0.611
84	0.55	0.125	0.5
85	1	0.5	1
86	1	1	1
87	1	0	1
88	1	0	1
89	0.881	0.571	0.643
90	1	0	1
91	1	0	1
92	1	0.5	1
93	1	0.5	1
94	1	0	1
95	1	0	1
96	1	1	0
97	1	0	1
98	1	0	1
99	1	0.75	1
100	1	0.333	1
101	0	0	0
102	1	0	1
103	1	0.375	0.938
104	1	0	1
105	0.25	1	0
106	0.444	0	0.333

download these results as csv

Task 1b Summary Results by Query

Simple Hit/Miss Counting [3]

MRR Method [4]

Task 1c, IOACAS2's dataset Results

As argued by Jang, there are some abnormal in the IOACAS2 data set. Therefore, we just provide the raw result without any further analysis.

Task 1c Summary Results by Query Group

Simple Hit/Miss Count

	JSSLP1	TY1	TY2
1	0	0	1
2	0	0	0
3	1	0	1
4	0	0	0
5	0.182	0.273	0.273
6	0	0	0
7	0	0.5	0.5
8	0	0	0
9	0	1	0
10	1	1	1
11	0	0	0.5
12	1	0	1
13	0	0.143	0.286
14	0.2	0.2	0.4
15	0	0	0
16	0	1	1
17	0	0	0
18	0	0	1
19	0	0	0.5
20	0	0.5	0
21	0	0	0.5
22	0	0	0
23	0	0	0
24	0	0	0
25	0	0	0
26	0	0	0.25
27	0	0	0
28	0	0	0
29	0.125	0.375	0.375
30	1	0	0
31	0	0	0
32	0	0	0
33	0	0.333	0.333
34	0	0	0
35	0	1	1
36	0	0.5	0
37	0.111	0	0.556
38	0	0.5	0.5
39	0	0	1
40	0.5	0.5	0.5
41	1	1	1
42	0	0	0
43	0	0	0
44	0.5	0.5	1
45	0	0	0
46	0	0	0
47	0	0	1
48	0	0	1
49	0	0	0
50	0.375	0.375	0.25
51	0	0.25	0.25
52	0	0.333	0.333
53	0	0	0.5
54	0.5	0	0.25
55	1	0	0
56	1	0.667	0.333
57	0.333	0	0.333
58	0	0	0
59	0	1	1
60	1	0	1
61	0	0	0.5
62	0	0	0
63	1	0	0
64	0.333	0.333	0.333
65	1	1	0
66	1	0	0
67	0.5	0	0
68	1	0.6	0.6
69	1	0	0.333
70	0.6	0	0.7
71	1	0	0
72	0.833	0.667	0.5
73	0	0	0
74	0.333	0	0
75	0.5	0	0
76	0	0	0
77	1	1	0
78	0.667	0.833	0.5
79	0	0	0
80	0.615	0.615	0.692
81	0.2	0	0
82	0	0	0
83	0.571	0.857	0.143
84	0	0	0
85	1	0	0
86	1	0	1
87	1	1	1
88	0.25	0.25	0.75
89	1	0	1
90	0	0	0
91	0.5	0.5	0.5
92	1	0.5	1
93	1	0.4	0.8
94	0	1	0
95	0	0	1
96	0.667	0.667	0.333
97	1	0	1
98	1	1	1
99	0	1	0
100	0	0	0.5
101	0	0	1
102	0.875	0.5	0.375
103	1	0	1
104	1	1	0
105	1	0	1
106	0	0	0
107	0.5	0	1
108	1	0	1
109	1	1	0.5
110	0	0	1
111	0	0	0
112	0	0	0
113	0	0	0
114	1	1	1
115	1	1	0
116	0	0.5	0
117	0	0	0
118	1	1	0
119	0	0.2	0.2
120	0	0	0
121	0	0	0
122	1	1	0
123	1	0	1
124	0	0	0
125	1	0.5	1
126	0	0	0
127	0	0	0
128	0	0	0
129	1	0	1
130	0	0	0
131	0	1	1
132	0	0	0.5
133	1	1	1
134	1	1	1
135	1	0	0
136	1	0	1
137	1	0	1
138	1	0	0.667
139	1	1	1
140	1	1	0
141	1	1	1
142	1	0	0.75
143	1	0	0
144	1	0	0
145	0.333	0.333	0.333
146	0	0	0
147	1	0	1
148	1	0.5	1
149	1	1	1
150	1	0.5	1
151	1	0	1
152	0.667	0.333	0.333
153	1	0	0
154	1	0	1
155	0	0	1
156	1	1	0
157	1	1	1
158	0	0	0
159	0	0	0
160	0	0	0
161	0	1	0
162	0	1	0
163	0	0	0
164	1	0	1
165	1	0	0
166	0	0	0
167	1	1	1
168	1	1	1
169	1	1	1
170	1	1	1
171	0	0	0
172	1	1	1
173	1	1	1
174	1	1	0
175	1	0	1
176	1	0	1
177	0	0	0
178	1	1	0.5
179	1	1	1
180	1	0	0
181	1	1	1
182	1	0	0
183	1	0	1
184	0	0	1
185	0	1	1
186	1	0	0.667
187	1	0	1
188	0	1	0
189	0	0	1
190	1	0.5	1
191	1	1	0
192	0	0	1

download these results as csv

MRR Method

	JSSLP1	TY1	TY2
1	0	0	1
2	0	0	0
3	1	0	1
4	0	0	0
5	0.025	0.132	0.133
6	0	0	0
7	0	0.5	0.083
8	0	0	0
9	0	1	0
10	1	1	1
11	0	0	0.3
12	1	0	1
13	0	0.143	0.163
14	0.04	0.022	0.073
15	0	0	0
16	0	1	0.5
17	0	0	0
18	0	0	0.2
19	0	0	0.071
20	0	0.092	0
21	0	0	0.063
22	0	0	0
23	0	0	0
24	0	0	0
25	0	0	0
26	0	0	0.042
27	0	0	0
28	0	0	0
29	0.042	0.375	0.292
30	0.333	0	0
31	0	0	0
32	0	0	0
33	0	0.167	0.083
34	0	0	0
35	0	0.1	1
36	0	0.5	0
37	0.037	0	0.347
38	0	0.5	0.167
39	0	0	0.5
40	0.5	0.5	0.25
41	1	0.1	0.5
42	0	0	0
43	0	0	0
44	0.5	0.5	0.75
45	0	0	0
46	0	0	0
47	0	0	0.25
48	0	0	0.5
49	0	0	0
50	0.156	0.153	0.104
51	0	0.025	0.25
52	0	0.333	0.111
53	0	0	0.5
54	0.5	0	0.031
55	1	0	0
56	1	0.25	0.167
57	0.333	0	0.042
58	0	0	0
59	0	1	1
60	1	0	1
61	0	0	0.25
62	0	0	0
63	1	0	0
64	0.333	0.333	0.235
65	0.25	0.5	0
66	1	0	0
67	0.167	0	0
68	0.808	0.511	0.448
69	1	0	0.167
70	0.513	0	0.7
71	1	0	0
72	0.833	0.542	0.357
73	0	0	0
74	0.063	0	0
75	0.125	0	0
76	0	0	0
77	1	0.333	0
78	0.431	0.597	0.274
79	0	0	0
80	0.492	0.547	0.564
81	0.1	0	0
82	0	0	0
83	0.214	0.619	0.048
84	0	0	0
85	1	0	0
86	1	0	0.813
87	0.143	1	0.333
88	0.25	0.25	0.625
89	1	0	1
90	0	0	0
91	0.5	0.5	0.5
92	1	0.5	1
93	0.867	0.4	0.65
94	0	0.143	0
95	0	0	0.143
96	0.667	0.667	0.333
97	1	0	0.6
98	1	1	0.583
99	0	1	0
100	0	0	0.167
101	0	0	0.1
102	0.688	0.5	0.375
103	1	0	1
104	0.143	1	0
105	1	0	1
106	0	0	0
107	0.063	0	1
108	1	0	0.333
109	0.75	1	0.5
110	0	0	0.5
111	0	0	0
112	0	0	0
113	0	0	0
114	1	1	1
115	1	0.75	0
116	0	0.5	0
117	0	0	0
118	1	0.333	0
119	0	0.1	0.2
120	0	0	0
121	0	0	0
122	0.143	1	0
123	1	0	0.333
124	0	0	0
125	1	0.063	1
126	0	0	0
127	0	0	0
128	0	0	0
129	0.111	0	1
130	0	0	0
131	0	1	1
132	0	0	0.167
133	0.5	1	1
134	1	1	1
135	0.5	0	0
136	1	0	1
137	0.143	0	0.143
138	0.833	0	0.278
139	1	1	1
140	1	0.5	0
141	1	1	1
142	0.708	0	0.165
143	1	0	0
144	1	0	0
145	0.333	0.333	0.333
146	0	0	0
147	0.25	0	0.5
148	1	0.5	1
149	1	1	1
150	0.571	0.5	0.5
151	1	0	0.5
152	0.444	0.111	0.333
153	1	0	0
154	0.25	0	0.5
155	0	0	0.2
156	0.2	1	0
157	1	1	1
158	0	0	0
159	0	0	0
160	0	0	0
161	0	0.25	0
162	0	0.333	0
163	0	0	0
164	0.333	0	1
165	1	0	0
166	0	0	0
167	0.5	1	1
168	1	1	0.5
169	1	1	1
170	1	1	0.167
171	0	0	0
172	1	1	1
173	1	1	1
174	1	0.111	0
175	1	0	0.5
176	1	0	0.5
177	0	0	0
178	0.625	1	0.063
179	1	1	1
180	0.5	0	0
181	1	1	1
182	1	0	0
183	1	0	1
184	0	0	1
185	0	0.125	0.333
186	1	0	0.5
187	1	0	0.333
188	0	0.125	0
189	0	0	1
190	1	0.5	0.417
191	1	0.5	0
192	0	0	1

download these results as csv

Task 1c Summary Results by Query

Simple Hit/Miss Counting [5]

MRR Method [6]

Task 2 Results

Task 2 Overall Results

	JSSLP1	TY1	TY2
Multi Count	9.275	8.744	8.744
Total Count	2040	2040	2040

download these results as csv

Task 2 Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 2 data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
JSSLP1	TY2	0.2107	0.6250	1.0393	TRUE
JSSLP1	TY1	0.2107	0.6250	1.0393	TRUE
TY2	TY1	-0.4143	0.0000	0.4143	FALSE

download these results as csv

Task 2 Summary Results by Query Group

	JSSLP1	TY1	TY2
1	5.889	5.333	5.333
2	2.000	2.250	2.250
3	4.000	0.000	0.000
4	3.200	4.400	4.400
5	7.000	6.800	6.800
6	3.000	5.000	5.000
7	2.800	6.600	6.600
8	8.600	6.800	6.800
9	3.500	2.000	2.000
10	5.889	4.556	4.556
11	9.349	8.791	8.791
12	9.222	9.542	9.542
13	9.145	9.237	9.237
14	9.625	8.722	8.722
15	8.071	5.214	5.214
16	9.757	8.932	8.932
17	9.167	9.181	9.181
18	9.789	9.026	9.026
19	9.478	9.333	9.333
20	9.659	9.341	9.341
21	8.735	8.971	8.971
22	9.663	9.398	9.398
23	9.581	8.516	8.516
24	8.898	9.136	9.136
25	8.898	8.119	8.119
26	9.095	7.619	7.619
27	9.267	8.156	8.156
28	9.152	9.043	9.043
29	9.956	9.426	9.426
30	9.418	9.687	9.687
31	9.800	9.650	9.650
32	9.484	9.719	9.719
33	8.980	7.824	7.824
34	9.721	9.456	9.456
35	9.710	8.971	8.971
36	7.167	7.833	7.833
37	8.893	7.250	7.250
38	9.741	8.593	8.593
39	9.662	7.574	7.574
40	9.481	8.815	8.815
41	9.136	9.322	9.322
42	9.071	7.119	7.119
43	9.483	8.483	8.483
44	8.778	9.361	9.361
45	9.574	7.298	7.298
46	9.368	8.368	8.368
47	8.769	8.231	8.231
48	8.636	8.364	8.364

download these results as csv

Task 2 Summary Results by Query

	JSSLP1	TY1	TY2
1	2	6	6
2	5	7	7
3	7	6	6
4	8	8	8
5	3	8	8
6	5	7	7
7	7	7	7
8	6	8	8
9	8	8	8
10	7	7	7
11	6	9	9
12	7	8	8
13	7	7	7
14	8	8	8
15	4	5	5
16	0	1	1
17	7	7	7
18	8	7	7
19	4	0	0
20	9	8	8
21	6	10	10
22	10	10	10
23	10	10	10
24	10	9	9
25	2	10	10
26	10	9	9
27	10	10	10
28	10	10	10
29	9	10	10
30	10	10	10
31	10	10	10
32	10	10	10
33	10	10	10
34	10	10	10
35	10	10	10
36	10	10	10
37	7	8	8
38	5	10	10
39	10	10	10
40	4	10	10
41	10	10	10
42	7	10	10
43	10	10	10
44	10	10	10
45	10	10	10
46	9	9	9
47	10	10	10
48	10	10	10
49	10	8	8
50	10	7	7
51	10	10	10
52	10	10	10
53	10	10	10
54	10	10	10
55	10	7	7
56	10	10	10
57	10	10	10
58	10	10	10
59	10	9	9
60	10	10	10
61	10	10	10
62	10	10	10
63	10	6	6
64	10	10	10
65	10	10	10
66	10	10	10
67	10	9	9
68	10	9	9
69	10	10	10
70	10	10	10
71	10	10	10
72	10	8	8
73	10	10	10
74	10	8	8
75	10	10	10
76	10	10	10
77	10	10	10
78	10	9	9
79	10	8	8
80	10	10	10
81	5	10	10
82	10	10	10
83	10	9	9
84	10	9	9
85	10	9	9
86	10	10	10
87	5	10	10
88	10	10	10
89	10	8	8
90	10	10	10
91	7	10	10
92	10	9	9
93	10	10	10
94	10	10	10
95	10	6	6
96	10	10	10
97	10	10	10
98	6	4	4
99	10	10	10
100	10	10	10
101	10	10	10
102	10	8	8
103	10	10	10
104	10	10	10
105	10	10	10
106	10	10	10
107	10	10	10
108	10	10	10
109	10	10	10
110	10	10	10
111	10	10	10
112	10	9	9
113	10	10	10
114	10	9	9
115	10	10	10
116	10	10	10
117	10	10	10
118	10	10	10
119	10	10	10
120	10	10	10
121	10	10	10
122	10	10	10
123	10	10	10
124	10	10	10
125	10	10	10
126	10	10	10
127	10	10	10
128	10	6	6
129	10	10	10
130	10	10	10
131	10	10	10
132	10	10	10
133	10	10	10
134	10	10	10
135	10	10	10
136	10	10	10
137	10	8	8
138	10	10	10
139	10	10	10
140	10	10	10
141	8	9	9
142	10	10	10
143	10	8	8
144	10	8	8
145	10	9	9
146	10	10	10
147	10	10	10
148	8	3	3
149	10	10	10
150	10	10	10
151	10	9	9
152	10	10	10
153	10	10	10
154	10	10	10
155	10	10	10
156	10	10	10
157	10	9	9
158	9	9	9
159	10	10	10
160	9	10	10
161	10	4	4
162	10	7	7
163	7	10	10
164	10	10	10
165	10	7	7
166	10	9	9
167	10	10	10
168	10	10	10
169	10	10	10
170	10	9	9
171	10	10	10
172	10	10	10
173	10	10	10
174	10	10	10
175	10	10	10
176	10	10	10
177	10	10	10
178	10	10	10
179	10	10	10
180	10	10	10
181	10	10	10
182	10	10	10
183	10	10	10
184	10	10	10
185	10	10	10
186	10	8	8
187	10	10	10
188	10	3	3
189	8	6	6
190	10	1	1
191	9	10	10
192	10	10	10
193	7	1	1
194	10	10	10
195	5	5	5
196	8	6	6
197	10	7	7
198	10	10	10
199	10	9	9
200	10	10	10
201	10	9	9
202	10	10	10
203	10	10	10
204	10	5	5
205	10	10	10
206	10	9	9
207	10	10	10
208	10	9	9
209	10	10	10
210	10	10	10
211	10	10	10
212	10	9	9
213	10	10	10
214	10	10	10
215	10	10	10
216	10	10	10
217	10	10	10
218	10	10	10
219	10	10	10
220	10	10	10
221	10	10	10
222	10	10	10
223	10	10	10
224	10	10	10
225	10	10	10
226	10	10	10
227	10	10	10
228	10	10	10
229	10	10	10
230	10	8	8
231	10	10	10
232	10	10	10
233	10	9	9
234	10	10	10
235	10	10	10
236	10	9	9
237	10	10	10
238	10	10	10
239	10	10	10
240	10	10	10
241	10	10	10
242	10	9	9
243	10	9	9
244	10	10	10
245	10	9	9
246	10	10	10
247	10	10	10
248	10	10	10
249	10	10	10
250	10	10	10
251	10	10	10
252	10	10	10
253	10	10	10
254	10	10	10
255	9	10	10
256	7	10	10
257	10	10	10
258	10	10	10
259	10	10	10
260	10	10	10
261	10	10	10
262	10	10	10
263	10	10	10
264	10	9	9
265	10	10	10
266	6	10	10
267	10	10	10
268	10	8	8
269	10	10	10
270	4	5	5
271	10	9	9
272	10	10	10
273	10	10	10
274	10	10	10
275	10	10	10
276	10	4	4
277	10	10	10
278	10	10	10
279	10	10	10
280	10	10	10
281	10	10	10
282	10	10	10
283	10	10	10
284	10	10	10
285	10	9	9
286	10	10	10
287	10	10	10
288	10	10	10
289	10	10	10
290	10	10	10
291	9	6	6
292	10	10	10
293	9	7	7
294	10	8	8
295	10	8	8
296	10	10	10
297	9	9	9
298	10	10	10
299	10	8	8
300	10	10	10
301	10	9	9
302	9	5	5
303	8	8	8
304	10	5	5
305	10	10	10
306	10	10	10
307	10	9	9
308	10	5	5
309	10	10	10
310	9	8	8
311	10	10	10
312	10	10	10
313	10	10	10
314	10	9	9
315	10	10	10
316	0	0	0
317	10	10	10
318	10	10	10
319	10	8	8
320	10	10	10
321	10	10	10
322	2	5	5
323	10	10	10
324	10	10	10
325	10	9	9
326	10	9	9
327	8	5	5
328	10	10	10
329	10	9	9
330	10	10	10
331	10	10	10
332	10	10	10
333	10	10	10
334	10	10	10
335	10	10	10
336	10	10	10
337	10	10	10
338	10	10	10
339	10	10	10
340	10	9	9
341	10	10	10
342	10	10	10
343	10	10	10
344	10	10	10
345	10	10	10
346	10	10	10
347	10	10	10
348	10	10	10
349	4	8	8
350	10	10	10
351	9	9	9
352	9	10	10
353	9	9	9
354	6	10	10
355	10	9	9
356	10	9	9
357	8	8	8
358	10	3	3
359	10	10	10
360	10	10	10
361	10	10	10
362	4	9	9
363	10	9	9
364	9	10	10
365	4	8	8
366	10	8	8
367	9	10	10
368	10	10	10
369	10	10	10
370	10	10	10
371	10	10	10
372	10	10	10
373	10	10	10
374	10	10	10
375	10	10	10
376	10	10	10
377	10	10	10
378	10	10	10
379	10	7	7
380	10	10	10
381	10	10	10
382	10	10	10
383	10	10	10
384	9	10	10
385	10	10	10
386	10	10	10
387	10	10	10
388	10	10	10
389	10	10	10
390	10	10	10
391	10	10	10
392	10	10	10
393	10	10	10
394	9	7	7
395	10	2	2
396	10	10	10
397	10	10	10
398	10	10	10
399	10	10	10
400	10	8	8
401	10	10	10
402	10	10	10
403	10	10	10
404	10	10	10
405	10	10	10
406	10	10	10
407	7	9	9
408	10	9	9
409	10	10	10
410	10	10	10
411	10	10	10
412	10	10	10
413	10	10	10
414	10	10	10
415	10	10	10
416	10	10	10
417	7	10	10
418	10	10	10
419	10	10	10
420	10	10	10
421	10	10	10
422	10	10	10
423	10	10	10
424	10	10	10
425	10	3	3
426	10	8	8
427	10	10	10
428	3	4	4
429	10	7	7
430	3	9	9
431	10	10	10
432	7	10	10
433	6	7	7
434	10	7	7
435	9	9	9
436	10	10	10
437	10	9	9
438	10	8	8
439	10	9	9
440	10	10	10
441	10	0	0
442	1	3	3
443	10	10	10
444	10	10	10
445	10	7	7
446	9	1	1
447	7	0	0
448	9	9	9
449	4	8	8
450	10	2	2
451	10	2	2
452	10	2	2
453	9	9	9
454	10	8	8
455	10	2	2
456	8	6	6
457	9	3	3
458	8	9	9
459	3	3	3
460	10	10	10
461	10	10	10
462	10	8	8
463	4	9	9
464	10	10	10
465	10	4	4
466	10	10	10
467	10	10	10
468	10	10	10
469	9	8	8
470	9	10	10
471	10	10	10
472	10	10	10
473	10	9	9
474	10	9	9
475	10	10	10
476	10	10	10
477	8	10	10
478	10	9	9
479	10	10	10
480	10	10	10
481	10	8	8
482	10	7	7
483	10	10	10
484	10	5	5
485	10	10	10
486	10	10	10
487	9	9	9
488	10	10	10
489	10	9	9
490	10	8	8
491	10	10	10
492	10	10	10
493	10	10	10
494	10	10	10
495	10	10	10
496	10	9	9
497	10	10	10
498	10	9	9
499	8	9	9
500	10	10	10
501	10	10	10
502	10	10	10
503	10	10	10
504	10	10	10
505	10	10	10
506	10	9	9
507	10	10	10
508	10	10	10
509	10	10	10
510	10	10	10
511	10	10	10
512	10	10	10
513	10	10	10
514	10	6	6
515	10	10	10
516	10	10	10
517	10	10	10
518	10	10	10
519	10	10	10
520	7	10	10
521	10	10	10
522	1	5	5
523	10	10	10
524	10	10	10
525	7	10	10
526	10	10	10
527	5	7	7
528	10	0	0
529	9	9	9
530	10	3	3
531	3	10	10
532	5	9	9
533	10	9	9
534	3	10	10
535	10	10	10
536	10	10	10
537	9	7	7
538	8	10	10
539	10	10	10
540	8	10	10
541	10	10	10
542	9	7	7
543	10	10	10
544	10	10	10
545	10	8	8
546	10	10	10
547	10	10	10
548	10	8	8
549	10	8	8
550	10	10	10
551	10	10	10
552	10	10	10
553	10	7	7
554	10	10	10
555	10	10	10
556	10	10	10
557	10	10	10
558	10	10	10
559	10	10	10
560	10	10	10
561	10	10	10
562	10	10	10
563	10	9	9
564	10	9	9
565	10	10	10
566	10	10	10
567	10	10	10
568	10	10	10
569	10	6	6
570	10	10	10
571	10	10	10
572	10	10	10
573	10	10	10
574	10	10	10
575	10	10	10
576	10	10	10
577	10	4	4
578	10	10	10
579	10	9	9
580	10	10	10
581	10	10	10
582	10	10	10
583	10	10	10
584	10	10	10
585	10	10	10
586	10	10	10
587	10	10	10
588	10	10	10
589	10	10	10
590	10	10	10
591	10	10	10
592	10	10	10
593	10	10	10
594	10	10	10
595	10	10	10
596	10	8	8
597	10	10	10
598	10	10	10
599	10	10	10
600	10	10	10
601	10	10	10
602	10	10	10
603	10	10	10
604	10	10	10
605	10	10	10
606	10	8	8
607	10	10	10
608	10	9	9
609	10	10	10
610	10	10	10
611	10	10	10
612	10	10	10
613	10	10	10
614	10	10	10
615	10	10	10
616	10	9	9
617	10	10	10
618	10	10	10
619	10	10	10
620	10	5	5
621	10	10	10
622	10	10	10
623	10	10	10
624	10	5	5
625	10	10	10
626	10	10	10
627	10	9	9
628	7	5	5
629	10	10	10
630	10	9	9
631	10	10	10
632	10	10	10
633	10	10	10
634	10	8	8
635	10	10	10
636	10	9	9
637	10	7	7
638	10	9	9
639	10	10	10
640	10	8	8
641	10	2	2
642	10	10	10
643	10	10	10
644	7	10	10
645	10	7	7
646	9	9	9
647	10	10	10
648	10	9	9
649	10	10	10
650	10	8	8
651	10	10	10
652	10	9	9
653	10	10	10
654	10	10	10
655	10	10	10
656	10	10	10
657	10	9	9
658	10	10	10
659	10	10	10
660	10	10	10
661	10	10	10
662	10	5	5
663	10	10	10
664	10	10	10
665	3	10	10
666	9	8	8
667	10	10	10
668	2	8	8
669	0	8	8
670	10	10	10
671	10	2	2
672	10	6	6
673	7	1	1
674	10	10	10
675	8	8	8
676	10	10	10
677	10	9	9
678	0	1	1
679	9	6	6
680	10	1	1
681	0	10	10
682	10	10	10
683	10	10	10
684	8	5	5
685	9	10	10
686	10	10	10
687	6	0	0
688	10	10	10
689	9	9	9
690	10	10	10
691	6	6	6
692	10	10	10
693	10	7	7
694	10	10	10
695	10	10	10
696	10	3	3
697	10	10	10
698	8	2	2
699	10	10	10
700	2	9	9
701	9	10	10
702	3	8	8
703	9	3	3
704	10	6	6
705	5	8	8
706	10	4	4
707	6	4	4
708	10	2	2
709	10	1	1
710	10	3	3
711	9	8	8
712	7	8	8
713	10	6	6
714	10	7	7
715	10	9	9
716	10	6	6
717	10	1	1
718	1	10	10
719	10	3	3
720	10	0	0
721	5	10	10
722	10	7	7
723	10	10	10
724	10	9	9
725	10	8	8
726	10	10	10
727	10	10	10
728	10	10	10
729	10	10	10
730	10	10	10
731	10	10	10
732	10	9	9
733	10	10	10
734	10	10	10
735	10	8	8
736	10	10	10
737	10	10	10
738	10	10	10
739	10	10	10
740	10	10	10
741	10	10	10
742	10	9	9
743	4	8	8
744	10	10	10
745	10	9	9
746	10	5	5
747	0	8	8
748	10	10	10
749	10	10	10
750	10	8	8
751	10	2	2
752	1	7	7
753	8	4	4
754	0	1	1
755	10	10	10
756	10	4	4
757	10	1	1
758	10	10	10
759	9	1	1
760	10	10	10
761	9	9	9
762	8	1	1
763	3	9	9
764	10	10	10
765	5	4	4
766	8	9	9
767	10	10	10
768	10	9	9
769	10	1	1
770	10	9	9
771	10	10	10
772	10	7	7
773	10	4	4
774	10	10	10
775	10	10	10
776	6	6	6
777	10	10	10
778	10	10	10
779	10	6	6
780	7	4	4
781	10	10	10
782	10	10	10
783	10	8	8
784	10	10	10
785	10	10	10
786	6	4	4
787	10	5	5
788	10	10	10
789	10	10	10
790	10	10	10
791	6	10	10
792	10	10	10
793	10	10	10
794	10	10	10
795	10	8	8
796	10	10	10
797	10	10	10
798	0	0	0
799	10	10	10
800	10	8	8
801	10	10	10
802	10	8	8
803	10	10	10
804	10	10	10
805	10	10	10
806	10	8	8
807	10	10	10
808	10	10	10
809	10	9	9
810	10	10	10
811	10	7	7
812	9	10	10
813	10	10	10
814	10	10	10
815	10	10	10
816	10	10	10
817	10	10	10
818	10	10	10
819	10	10	10
820	10	10	10
821	10	10	10
822	8	9	9
823	9	10	10
824	10	10	10
825	10	10	10
826	10	10	10
827	10	10	10
828	10	10	10
829	10	7	7
830	10	7	7
831	10	10	10
832	10	6	6
833	10	10	10
834	10	10	10
835	10	10	10
836	10	10	10
837	10	10	10
838	10	3	3
839	10	10	10
840	10	10	10
841	10	10	10
842	10	10	10
843	10	10	10
844	10	10	10
845	10	10	10
846	10	10	10
847	10	10	10
848	10	10	10
849	10	7	7
850	10	10	10
851	10	9	9
852	10	10	10
853	10	10	10
854	10	8	8
855	10	10	10
856	10	10	10
857	10	10	10
858	10	10	10
859	10	10	10
860	10	10	10
861	10	10	10
862	10	10	10
863	10	8	8
864	10	9	9
865	10	10	10
866	10	10	10
867	10	10	10
868	10	10	10
869	10	8	8
870	10	10	10
871	10	9	9
872	10	10	10
873	10	10	10
874	10	9	9
875	10	10	10
876	10	10	10
877	10	10	10
878	10	10	10
879	10	10	10
880	10	10	10
881	10	10	10
882	10	10	10
883	10	10	10
884	10	10	10
885	10	10	10
886	10	10	10
887	10	10	10
888	10	10	10
889	10	10	10
890	9	9	9
891	10	10	10
892	10	10	10
893	10	8	8
894	10	10	10
895	8	10	10
896	10	10	10
897	10	10	10
898	10	10	10
899	10	10	10
900	10	10	10
901	8	10	10
902	10	10	10
903	10	10	10
904	10	4	4
905	10	10	10
906	2	10	10
907	10	10	10
908	10	10	10
909	6	4	4
910	10	10	10
911	8	8	8
912	10	10	10
913	10	9	9
914	10	10	10
915	9	3	3
916	10	10	10
917	10	10	10
918	10	10	10
919	10	10	10
920	10	10	10
921	10	10	10
922	10	10	10
923	10	8	8
924	10	7	7
925	10	10	10
926	10	9	9
927	10	9	9
928	7	1	1
929	10	10	10
930	10	10	10
931	10	10	10
932	10	9	9
933	9	8	8
934	10	10	10
935	10	10	10
936	10	10	10
937	10	10	10
938	10	10	10
939	10	8	8
940	10	10	10
941	10	5	5
942	10	10	10
943	10	10	10
944	10	10	10
945	10	9	9
946	10	10	10
947	10	10	10
948	10	10	10
949	10	10	10
950	10	10	10
951	10	8	8
952	10	10	10
953	10	10	10
954	10	10	10
955	10	10	10
956	10	10	10
957	10	10	10
958	10	5	5
959	9	8	8
960	10	10	10
961	9	10	10
962	8	9	9
963	10	10	10
964	0	10	10
965	10	10	10
966	10	10	10
967	10	10	10
968	10	10	10
969	10	10	10
970	10	10	10
971	10	10	10
972	10	9	9
973	10	10	10
974	10	10	10
975	10	6	6
976	10	10	10
977	10	10	10
978	8	9	9
979	10	10	10
980	10	0	0
981	10	10	10
982	10	9	9
983	10	4	4
984	10	6	6
985	10	8	8
986	10	10	10
987	10	10	10
988	2	0	0
989	8	1	1
990	10	8	8
991	10	10	10
992	10	7	7
993	10	10	10
994	10	10	10
995	10	10	10
996	4	9	9
997	10	1	1
998	10	10	10
999	10	10	10
1000	10	10	10
1001	10	4	4
1002	10	9	9
1003	10	10	10
1004	10	10	10
1005	10	7	7
1006	9	9	9
1007	10	10	10
1008	6	5	5
1009	8	7	7
1010	10	1	1
1011	10	10	10
1012	10	10	10
1013	10	10	10
1014	10	8	8
1015	10	9	9
1016	10	9	9
1017	10	6	6
1018	7	8	8
1019	10	9	9
1020	10	10	10
1021	9	8	8
1022	10	10	10
1023	10	9	9
1024	7	10	10
1025	10	9	9
1026	9	10	10
1027	10	10	10
1028	10	9	9
1029	10	9	9
1030	2	10	10
1031	10	10	10
1032	10	10	10
1033	10	10	10
1034	10	7	7
1035	10	10	10
1036	10	10	10
1037	10	8	8
1038	9	9	9
1039	10	10	10
1040	10	8	8
1041	10	10	10
1042	10	10	10
1043	10	9	9
1044	10	10	10
1045	10	10	10
1046	10	10	10
1047	10	7	7
1048	10	10	10
1049	10	10	10
1050	10	10	10
1051	10	10	10
1052	10	10	10
1053	10	10	10
1054	10	10	10
1055	10	10	10
1056	10	10	10
1057	10	10	10
1058	6	9	9
1059	10	5	5
1060	10	10	10
1061	10	8	8
1062	10	9	9
1063	10	10	10
1064	10	10	10
1065	9	4	4
1066	10	1	1
1067	10	10	10
1068	10	10	10
1069	10	8	8
1070	10	10	10
1071	10	10	10
1072	10	10	10
1073	6	3	3
1074	10	10	10
1075	10	10	10
1076	10	8	8
1077	9	9	9
1078	10	10	10
1079	10	7	7
1080	8	10	10
1081	10	10	10
1082	10	10	10
1083	10	10	10
1084	10	10	10
1085	5	10	10
1086	10	10	10
1087	10	10	10
1088	10	10	10
1089	10	10	10
1090	2	10	10
1091	10	8	8
1092	8	10	10
1093	6	10	10
1094	10	10	10
1095	6	10	10
1096	6	10	10
1097	10	10	10
1098	9	10	10
1099	10	8	8
1100	10	10	10
1101	10	10	10
1102	10	10	10
1103	10	10	10
1104	10	10	10
1105	10	10	10
1106	10	10	10
1107	10	10	10
1108	10	10	10
1109	10	10	10
1110	10	10	10
1111	10	10	10
1112	10	10	10
1113	10	10	10
1114	10	10	10
1115	10	10	10
1116	10	10	10
1117	10	10	10
1118	10	10	10
1119	10	10	10
1120	10	10	10
1121	10	10	10
1122	10	8	8
1123	10	10	10
1124	10	10	10
1125	10	8	8
1126	10	10	10
1127	10	10	10
1128	10	10	10
1129	10	7	7
1130	9	10	10
1131	5	9	9
1132	10	9	9
1133	10	10	10
1134	10	10	10
1135	10	8	8
1136	10	10	10
1137	8	9	9
1138	10	9	9
1139	10	9	9
1140	10	10	10
1141	10	5	5
1142	10	7	7
1143	10	5	5
1144	10	10	10
1145	10	8	8
1146	9	9	9
1147	10	10	10
1148	8	10	10
1149	10	6	6
1150	5	8	8
1151	10	8	8
1152	10	0	0
1153	8	7	7
1154	5	9	9
1155	10	10	10
1156	10	10	10
1157	10	0	0
1158	10	10	10
1159	10	10	10
1160	1	8	8
1161	10	10	10
1162	10	10	10
1163	10	10	10
1164	9	10	10
1165	10	10	10
1166	10	5	5
1167	10	10	10
1168	10	10	10
1169	10	10	10
1170	10	10	10
1171	10	10	10
1172	10	10	10
1173	10	10	10
1174	10	10	10
1175	10	10	10
1176	10	10	10
1177	10	10	10
1178	10	10	10
1179	10	10	10
1180	10	10	10
1181	10	10	10
1182	10	4	4
1183	10	8	8
1184	5	10	10
1185	5	6	6
1186	10	4	4
1187	10	10	10
1188	10	10	10
1189	10	10	10
1190	10	8	8
1191	10	1	1
1192	4	9	9
1193	10	10	10
1194	10	10	10
1195	10	10	10
1196	10	10	10
1197	10	10	10
1198	10	9	9
1199	10	10	10
1200	10	4	4
1201	3	1	1
1202	10	10	10
1203	10	10	10
1204	10	10	10
1205	1	0	0
1206	0	0	0
1207	3	10	10
1208	1	0	0
1209	8	8	8
1210	0	3	3
1211	1	1	1
1212	10	7	7
1213	7	9	9
1214	1	2	2
1215	0	0	0
1216	4	1	1
1217	0	2	2
1218	4	1	1
1219	7	3	3
1220	10	5	5
1221	10	10	10
1222	0	1	1
1223	8	10	10
1224	4	10	10
1225	10	9	9
1226	0	0	0
1227	0	0	0
1228	7	0	0
1229	10	10	10
1230	10	8	8
1231	10	10	10
1232	10	9	9
1233	10	10	10
1234	10	10	10
1235	10	10	10
1236	10	10	10
1237	10	10	10
1238	5	10	10
1239	10	10	10
1240	10	10	10
1241	10	10	10
1242	10	10	10
1243	10	10	10
1244	10	10	10
1245	10	9	9
1246	10	10	10
1247	10	10	10
1248	10	8	8
1249	10	10	10
1250	10	10	10
1251	0	1	1
1252	5	9	9
1253	5	3	3
1254	10	7	7
1255	0	1	1
1256	0	0	0
1257	7	1	1
1258	3	0	0
1259	5	4	4
1260	1	0	0
1261	10	1	1
1262	2	1	1
1263	10	5	5
1264	0	0	0
1265	10	10	10
1266	5	2	2
1267	3	0	0
1268	6	5	5
1269	9	4	4
1270	7	2	2
1271	7	10	10
1272	10	10	10
1273	10	8	8
1274	7	9	9
1275	9	10	10
1276	10	2	2
1277	10	9	9
1278	10	10	10
1279	10	10	10
1280	10	10	10
1281	10	10	10
1282	10	10	10
1283	10	10	10
1284	10	9	9
1285	10	8	8
1286	10	10	10
1287	10	9	9
1288	10	10	10
1289	10	10	10
1290	10	10	10
1291	10	10	10
1292	10	6	6
1293	10	10	10
1294	10	10	10
1295	9	10	10
1296	10	9	9
1297	10	8	8
1298	10	10	10
1299	10	10	10
1300	10	9	9
1301	10	10	10
1302	10	10	10
1303	10	10	10
1304	10	10	10
1305	10	6	6
1306	10	10	10
1307	10	10	10
1308	9	2	2
1309	10	10	10
1310	10	10	10
1311	10	3	3
1312	10	10	10
1313	9	9	9
1314	10	2	2
1315	10	10	10
1316	10	9	9
1317	10	10	10
1318	10	8	8
1319	10	8	8
1320	10	10	10
1321	10	10	10
1322	10	10	10
1323	10	10	10
1324	10	10	10
1325	10	8	8
1326	10	10	10
1327	10	10	10
1328	10	10	10
1329	10	10	10
1330	10	10	10
1331	10	10	10
1332	10	10	10
1333	10	10	10
1334	10	6	6
1335	10	10	10
1336	10	9	9
1337	10	5	5
1338	10	10	10
1339	10	10	10
1340	10	9	9
1341	10	10	10
1342	6	4	4
1343	10	8	8
1344	10	9	9
1345	10	8	8
1346	10	9	9
1347	10	10	10
1348	10	10	10
1349	10	10	10
1350	10	10	10
1351	10	10	10
1352	10	10	10
1353	10	4	4
1354	10	10	10
1355	10	6	6
1356	10	10	10
1357	10	10	10
1358	10	10	10
1359	10	8	8
1360	9	8	8
1361	10	10	10
1362	10	10	10
1363	10	10	10
1364	10	10	10
1365	10	10	10
1366	10	10	10
1367	4	4	4
1368	10	9	9
1369	10	10	10
1370	10	10	10
1371	10	9	9
1372	10	10	10
1373	10	10	10
1374	10	10	10
1375	10	7	7
1376	10	10	10
1377	10	9	9
1378	9	9	9
1379	10	10	10
1380	10	2	2
1381	5	7	7
1382	10	9	9
1383	9	9	9
1384	6	2	2
1385	10	1	1
1386	10	3	3
1387	10	10	10
1388	3	5	5
1389	5	2	2
1390	10	8	8
1391	10	9	9
1392	10	3	3
1393	10	9	9
1394	2	10	10
1395	10	9	9
1396	10	10	10
1397	10	9	9
1398	10	9	9
1399	10	9	9
1400	10	9	9
1401	10	9	9
1402	9	9	9
1403	10	2	2
1404	10	10	10
1405	10	1	1
1406	10	8	8
1407	10	10	10
1408	10	10	10
1409	10	9	9
1410	10	10	10
1411	10	9	9
1412	10	10	10
1413	10	10	10
1414	10	10	10
1415	10	1	1
1416	10	10	10
1417	10	10	10
1418	10	8	8
1419	10	10	10
1420	10	10	10
1421	1	6	6
1422	10	10	10
1423	10	10	10
1424	10	10	10
1425	10	10	10
1426	10	10	10
1427	10	10	10
1428	10	10	10
1429	10	10	10
1430	10	10	10
1431	10	10	10
1432	10	10	10
1433	10	10	10
1434	10	10	10
1435	10	10	10
1436	10	10	10
1437	10	10	10
1438	10	9	9
1439	10	10	10
1440	10	10	10
1441	10	10	10
1442	10	10	10
1443	9	4	4
1444	10	10	10
1445	1	10	10
1446	10	8	8
1447	10	10	10
1448	10	10	10
1449	10	10	10
1450	10	10	10
1451	10	9	9
1452	10	10	10
1453	10	7	7
1454	10	10	10
1455	10	5	5
1456	6	2	2
1457	10	10	10
1458	10	10	10
1459	10	10	10
1460	10	9	9
1461	10	10	10
1462	10	8	8
1463	10	8	8
1464	10	10	10
1465	10	6	6
1466	10	7	7
1467	10	10	10
1468	10	10	10
1469	10	10	10
1470	10	9	9
1471	10	10	10
1472	10	10	10
1473	10	7	7
1474	10	4	4
1475	10	10	10
1476	10	9	9
1477	10	10	10
1478	10	10	10
1479	10	10	10
1480	10	10	10
1481	10	10	10
1482	10	10	10
1483	10	9	9
1484	10	10	10
1485	10	10	10
1486	10	7	7
1487	10	10	10
1488	10	10	10
1489	10	10	10
1490	10	10	10
1491	10	3	3
1492	9	10	10
1493	10	10	10
1494	10	9	9
1495	10	10	10
1496	10	9	9
1497	10	9	9
1498	10	10	10
1499	10	10	10
1500	10	10	10
1501	10	9	9
1502	10	1	1
1503	10	9	9
1504	10	5	5
1505	10	10	10
1506	10	10	10
1507	10	10	10
1508	10	9	9
1509	10	10	10
1510	10	10	10
1511	10	10	10
1512	10	7	7
1513	10	10	10
1514	10	10	10
1515	0	0	0
1516	10	8	8
1517	10	10	10
1518	5	2	2
1519	10	10	10
1520	10	10	10
1521	10	10	10
1522	2	1	1
1523	10	8	8
1524	10	10	10
1525	10	10	10
1526	10	10	10
1527	10	10	10
1528	10	6	6
1529	10	9	9
1530	10	9	9
1531	10	10	10
1532	10	7	7
1533	10	8	8
1534	10	10	10
1535	10	10	10
1536	10	8	8
1537	10	8	8
1538	10	10	10
1539	1	0	0
1540	0	9	9
1541	10	10	10
1542	10	10	10
1543	0	0	0
1544	10	9	9
1545	3	10	10
1546	10	10	10
1547	10	10	10
1548	10	10	10
1549	9	6	6
1550	10	10	10
1551	10	10	10
1552	10	10	10
1553	8	4	4
1554	10	9	9
1555	6	10	10
1556	10	7	7
1557	5	9	9
1558	10	10	10
1559	10	10	10
1560	10	10	10
1561	10	10	10
1562	10	9	9
1563	10	10	10
1564	10	10	10
1565	10	10	10
1566	10	10	10
1567	10	9	9
1568	10	10	10
1569	10	10	10
1570	10	10	10
1571	10	10	10
1572	10	10	10
1573	10	10	10
1574	10	9	9
1575	10	10	10
1576	10	10	10
1577	10	10	10
1578	10	10	10
1579	9	10	10
1580	9	9	9
1581	0	6	6
1582	10	7	7
1583	10	10	10
1584	0	1	1
1585	9	10	10
1586	10	9	9
1587	5	10	10
1588	10	9	9
1589	6	0	0
1590	1	3	3
1591	10	10	10
1592	10	6	6
1593	3	8	8
1594	5	1	1
1595	8	8	8
1596	2	6	6
1597	10	7	7
1598	10	10	10
1599	10	8	8
1600	10	10	10
1601	10	10	10
1602	1	0	0
1603	10	10	10
1604	10	10	10
1605	10	8	8
1606	10	10	10
1607	10	10	10
1608	10	10	10
1609	10	9	9
1610	10	5	5
1611	10	10	10
1612	10	8	8
1613	10	10	10
1614	8	4	4
1615	10	10	10
1616	8	10	10
1617	10	10	10
1618	10	10	10
1619	10	10	10
1620	10	10	10
1621	3	10	10
1622	10	9	9
1623	7	3	3
1624	10	10	10
1625	10	10	10
1626	0	1	1
1627	10	9	9
1628	10	10	10
1629	9	7	7
1630	10	10	10
1631	8	8	8
1632	0	0	0
1633	10	2	2
1634	0	0	0
1635	10	10	10
1636	10	9	9
1637	10	10	10
1638	10	6	6
1639	10	10	10
1640	10	10	10
1641	10	10	10
1642	9	7	7
1643	10	10	10
1644	0	10	10
1645	10	8	8
1646	10	10	10
1647	10	10	10
1648	10	10	10
1649	10	10	10
1650	0	0	0
1651	0	0	0
1652	0	0	0
1653	0	0	0
1654	2	0	0
1655	0	0	0
1656	10	10	10
1657	10	9	9
1658	10	10	10
1659	10	10	10
1660	10	10	10
1661	10	10	10
1662	10	10	10
1663	10	2	2
1664	10	10	10
1665	10	10	10
1666	10	10	10
1667	10	10	10
1668	10	10	10
1669	10	10	10
1670	10	10	10
1671	10	10	10
1672	10	10	10
1673	10	10	10
1674	10	10	10
1675	10	10	10
1676	10	10	10
1677	10	10	10
1678	10	10	10
1679	10	10	10
1680	10	10	10
1681	10	10	10
1682	10	10	10
1683	10	10	10
1684	10	10	10
1685	10	10	10
1686	10	8	8
1687	10	10	10
1688	10	9	9
1689	10	10	10
1690	10	10	10
1691	10	10	10
1692	10	10	10
1693	10	8	8
1694	9	9	9
1695	10	9	9
1696	10	10	10
1697	10	10	10
1698	10	10	10
1699	10	10	10
1700	10	9	9
1701	10	9	9
1702	10	10	10
1703	10	10	10
1704	10	10	10
1705	10	10	10
1706	2	10	10
1707	10	10	10
1708	10	10	10
1709	10	10	10
1710	10	10	10
1711	10	10	10
1712	10	10	10
1713	10	8	8
1714	10	10	10
1715	10	10	10
1716	10	10	10
1717	10	9	9
1718	10	10	10
1719	10	10	10
1720	10	10	10
1721	10	10	10
1722	10	9	9
1723	10	10	10
1724	10	7	7
1725	5	9	9
1726	10	9	9
1727	10	10	10
1728	10	10	10
1729	10	10	10
1730	10	8	8
1731	10	10	10
1732	10	10	10
1733	8	10	10
1734	10	7	7
1735	10	10	10
1736	8	10	10
1737	10	3	3
1738	10	10	10
1739	10	10	10
1740	8	9	9
1741	10	9	9
1742	10	8	8
1743	10	10	10
1744	10	8	8
1745	10	9	9
1746	10	10	10
1747	10	10	10
1748	10	10	10
1749	10	10	10
1750	10	10	10
1751	10	9	9
1752	10	8	8
1753	10	10	10
1754	10	10	10
1755	10	10	10
1756	10	10	10
1757	10	10	10
1758	10	9	9
1759	10	10	10
1760	10	9	9
1761	10	10	10
1762	10	10	10
1763	10	7	7
1764	10	4	4
1765	10	9	9
1766	2	2	2
1767	10	10	10
1768	10	6	6
1769	10	9	9
1770	8	9	9
1771	10	1	1
1772	10	9	9
1773	10	10	10
1774	10	5	5
1775	9	7	7
1776	10	10	10
1777	10	10	10
1778	10	10	10
1779	10	10	10
1780	10	5	5
1781	10	10	10
1782	9	8	8
1783	10	10	10
1784	10	10	10
1785	8	9	9
1786	10	10	10
1787	10	8	8
1788	10	10	10
1789	10	9	9
1790	10	10	10
1791	10	10	10
1792	10	10	10
1793	10	10	10
1794	10	10	10
1795	10	9	9
1796	10	0	0
1797	10	10	10
1798	10	10	10
1799	10	10	10
1800	10	10	10
1801	10	10	10
1802	10	9	9
1803	7	8	8
1804	7	10	10
1805	10	8	8
1806	10	10	10
1807	10	10	10
1808	8	8	8
1809	9	5	5
1810	10	9	9
1811	10	10	10
1812	10	10	10
1813	10	10	10
1814	10	9	9
1815	10	7	7
1816	10	10	10
1817	10	10	10
1818	9	8	8
1819	10	9	9
1820	10	10	10
1821	10	9	9
1822	10	10	10
1823	10	4	4
1824	10	10	10
1825	10	6	6
1826	10	9	9
1827	10	10	10
1828	10	5	5
1829	10	9	9
1830	10	10	10
1831	10	10	10
1832	10	10	10
1833	10	8	8
1834	10	9	9
1835	10	4	4
1836	10	10	10
1837	10	10	10
1838	10	10	10
1839	10	10	10
1840	10	10	10
1841	10	9	9
1842	10	10	10
1843	10	10	10
1844	10	9	9
1845	10	10	10
1846	10	2	2
1847	10	10	10
1848	10	4	4
1849	10	9	9
1850	10	9	9
1851	10	4	4
1852	10	10	10
1853	9	10	10
1854	10	1	1
1855	10	9	9
1856	10	9	9
1857	10	10	10
1858	10	10	10
1859	10	10	10
1860	10	1	1
1861	10	10	10
1862	10	8	8
1863	10	2	2
1864	10	10	10
1865	7	8	8
1866	10	9	9
1867	10	10	10
1868	10	10	10
1869	10	6	6
1870	10	10	10
1871	10	6	6
1872	10	10	10
1873	10	10	10
1874	10	10	10
1875	10	6	6
1876	10	9	9
1877	9	8	8
1878	10	10	10
1879	10	10	10
1880	10	10	10
1881	10	8	8
1882	10	9	9
1883	10	10	10
1884	10	10	10
1885	10	10	10
1886	10	9	9
1887	10	10	10
1888	10	8	8
1889	10	10	10
1890	10	10	10
1891	10	10	10
1892	10	10	10
1893	10	9	9
1894	10	9	9
1895	7	10	10
1896	10	10	10
1897	10	10	10
1898	10	10	10
1899	10	0	0
1900	10	10	10
1901	10	10	10
1902	10	10	10
1903	10	10	10
1904	10	10	10
1905	10	10	10
1906	10	10	10
1907	10	10	10
1908	10	4	4
1909	10	10	10
1910	10	10	10
1911	10	10	10
1912	0	10	10
1913	10	1	1
1914	10	10	10
1915	8	9	9
1916	10	10	10
1917	10	10	10
1918	10	10	10
1919	10	10	10
1920	10	8	8
1921	10	10	10
1922	10	10	10
1923	10	10	10
1924	10	6	6
1925	10	10	10
1926	10	10	10
1927	2	6	6
1928	10	10	10
1929	10	10	10
1930	10	10	10
1931	10	10	10
1932	10	8	8
1933	10	10	10
1934	10	10	10
1935	10	10	10
1936	10	10	10
1937	10	10	10
1938	10	10	10
1939	10	10	10
1940	10	10	10
1941	10	10	10
1942	10	10	10
1943	10	10	10
1944	10	10	10
1945	10	9	9
1946	10	10	10
1947	10	7	7
1948	10	10	10
1949	10	10	10
1950	10	10	10
1951	10	9	9
1952	10	10	10
1953	10	10	10
1954	10	5	5
1955	3	9	9
1956	10	10	10
1957	10	9	9
1958	10	10	10
1959	10	10	10
1960	10	10	10
1961	10	10	10
1962	8	9	9
1963	10	10	10
1964	10	10	10
1965	10	10	10
1966	10	10	10
1967	10	10	10
1968	10	10	10
1969	10	10	10
1970	10	9	9
1971	10	10	10
1972	10	8	8
1973	10	10	10
1974	10	10	10
1975	10	10	10
1976	9	10	10
1977	10	10	10
1978	9	10	10
1979	3	4	4
1980	10	9	9
1981	1	3	3
1982	10	9	9
1983	10	10	10
1984	10	10	10
1985	3	9	9
1986	6	8	8
1987	10	10	10
1988	6	6	6
1989	9	6	6
1990	10	10	10
1991	9	9	9
1992	7	9	9
1993	10	10	10
1994	5	10	10
1995	7	7	7
1996	3	10	10
1997	10	10	10
1998	10	10	10
1999	6	10	10
2000	10	9	9
2001	10	10	10
2002	10	9	9
2003	8	10	10
2004	10	10	10
2005	9	9	9
2006	10	10	10
2007	4	10	10
2008	8	10	10
2009	1	3	3
2010	10	10	10
2011	10	10	10
2012	10	10	10
2013	3	7	7
2014	4	8	8
2015	10	10	10
2016	10	8	8
2017	9	8	8
2018	10	10	10
2019	10	9	9
2020	10	9	9
2021	10	10	10
2022	10	10	10
2023	10	9	9
2024	10	10	10
2025	10	9	9
2026	10	10	10
2027	10	10	10
2028	10	9	9
2029	10	10	10
2030	10	10	10
2031	10	10	10
2032	10	10	10
2033	10	10	10
2034	10	4	4
2035	10	10	10
2036	10	10	10
2037	10	10	10
2038	10	10	10
2039	10	10	10
2040	10	8	8

download these results as csv

Runtime Results

file /nema-raid/www/mirex/results/2011/qbsh/QbshRunTime.csv not found