2009:Query-by-Singing/Humming Results

Introduction

These are the results for the 2008 running of the Query-by-Singing/Humming task. For background information about this task set please refer to the 2009:Query by Singing/Humming page.

Task Descriptions

Task 1 Goto Task 1 Results: The first subtask is the same as last year. In this subtask, submitted systems take a sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth, as well as the simple hit(1)/miss(0) counting, is calculated over the top 10 returns. Two data sets are used:

• Jang's Dataset Results Roger Jang's MIR-QBSH corpus with 48 songs as ground truth + 2000 Essen Collection MIDI noise files. See ESAC Data Homepage for more information about the Essen Collection. The queries consists of 4431 humming. All queries are from the beginning of references
• ThinkIT's Dataset Results IOACAS corpus 1 data set with 106 songs as ground truth + 2000 Essen Collection MIDI noise files. See ESAC Data Homepage for more information about the Essen Collection. The queries consists of 355 humming. There are no "singing from beginning" gurantee.
• IOACAS's 2nd Dataset Results IOACAS corpus 2 data set with 192 songs as ground truth + 2000 Essen Collection MIDI noise files. See ESAC Data Homepage for more information about the Essen Collection. The queries consists of 404 humming. There are no "singing from beginning" gurantee.

Task 2 Goto Task 2 Results: The second subtask is the query against other humming. In the second subtask, Roger Jang's MIR-QBSH corpus has been divided into two groups (2040 as queries and 2391 as database). The query is performed against the other humming database and the top 10 closed are returned. The score is simple count of how many returns belong to the same ground truth song.

General Legend

Team ID

CSJ1 = Jyh-Shing Roger Jang matched by beginning of the sond
CSJ2 = Jyh-Shing Roger Jang matched by anywhere of the song
HAFR = Pierre Hanna, Julien Allali, Pascal Ferraro,Matthias Robine

Task 1 Results

Task 1a, Jang's dataset Results

Task 1a Overall Results

	CSJ1	CSJ2	HAFR
Simple Count	0.94	0.9	0.77
MRR	0.91	0.86	0.66
Total count	4431	4431	4409

download these results as csv

Task 1a Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

Simple Hit/Miss Count:

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
CSJ1	CSJ2	0.3232	0.7812	1.2393	TRUE
CSJ1	HAFR	1.1670	1.6250	2.0830	TRUE
CSJ2	HAFR	0.3857	0.8438	1.3018	TRUE

download these results as csv

MRR Method:

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
CSJ1	CSJ2	0.2009	0.6667	1.1324	TRUE
CSJ1	HAFR	1.1801	1.6458	2.1116	TRUE
CSJ2	HAFR	0.5134	0.9792	1.4449	TRUE

download these results as csv

Task 1a Summary Results by Query Group

Simple Hit/Miss Count

	CSJ1	CSJ2	HAFR
1	0.89	0.84	0.89
2	0.71	0.5	0.36
3	0.64	0.64	0.64
4	0.8	0.8	0.67
5	0.93	0.73	0.87
6	0.92	0.75	0.67
7	0.93	0.87	0.47
8	0.93	0.93	0.6
9	0.92	0.92	0.73
10	0.74	0.84	0.84
11	0.94	0.89	0.8
12	0.96	0.95	0.92
13	0.98	0.97	0.88
14	0.98	0.97	0.92
15	0.96	0.96	0.78
16	0.93	0.9	0.93
17	0.97	0.85	0.73
18	0.99	0.96	0.75
19	0.97	0.94	0.89
20	0.99	0.96	0.46
21	0.88	0.79	0.32
22	0.97	0.97	0.95
23	0.98	0.98	0.85
24	0.83	0.82	0.58
25	0.72	0.45	0.57
26	0.98	0.92	0.65
27	0.95	0.98	0.76
28	0.87	0.69	0.49
29	0.98	0.97	0.57
30	0.97	0.94	0.94
31	0.97	0.95	0.94
32	0.96	0.93	0.8
33	0.92	0.86	0.5
34	0.93	0.91	0.64
35	0.95	0.87	0.76
36	0.93	0.91	0.91
37	0.92	0.91	0.88
38	0.95	0.95	0.78
39	0.89	0.83	0.82
40	0.94	0.9	0.44
41	0.94	0.93	0.9
42	0.96	0.93	0.86
43	0.95	0.94	0.9
44	0.93	0.94	0.88
45	0.95	0.91	0.83
46	0.95	0.92	0.8
47	1	0.97	0.93
48	1	0.96	0.81

download these results as csv

MRR Method

	CSJ1	CSJ2	HAFR
1	0.82	0.77	0.82
2	0.61	0.5	0.36
3	0.64	0.64	0.58
4	0.8	0.8	0.62
5	0.82	0.7	0.72
6	0.79	0.62	0.59
7	0.89	0.87	0.47
8	0.88	0.93	0.52
9	0.88	0.83	0.64
10	0.7	0.84	0.72
11	0.92	0.85	0.72
12	0.94	0.94	0.79
13	0.96	0.94	0.74
14	0.95	0.9	0.81
15	0.94	0.96	0.7
16	0.9	0.89	0.9
17	0.91	0.7	0.56
18	0.97	0.93	0.64
19	0.96	0.91	0.8
20	0.98	0.95	0.3
21	0.75	0.69	0.24
22	0.95	0.95	0.91
23	0.96	0.96	0.71
24	0.78	0.75	0.49
25	0.58	0.35	0.4
26	0.97	0.9	0.57
27	0.95	0.95	0.58
28	0.74	0.58	0.38
29	0.96	0.95	0.43
30	0.96	0.9	0.88
31	0.97	0.94	0.84
32	0.93	0.89	0.64
33	0.88	0.81	0.36
34	0.91	0.87	0.53
35	0.93	0.86	0.67
36	0.93	0.91	0.84
37	0.87	0.86	0.8
38	0.9	0.91	0.63
39	0.86	0.79	0.69
40	0.89	0.85	0.31
41	0.91	0.91	0.82
42	0.94	0.91	0.76
43	0.91	0.85	0.8
44	0.92	0.93	0.84
45	0.93	0.9	0.77
46	0.92	0.9	0.74
47	0.98	0.94	0.83
48	0.95	0.95	0.7

download these results as csv

Task 1a Summary Results by Query

Simple Hit/Miss Counting [1]

MRR Method [2]

Task 1b, ThinkIT's dataset Results

Task 1b Overall Results

	CSJ1	CSJ2	HAFR
Simple Count	0.43	0.86	0.78
MRR	0.41	0.8	0.68
Total count	355	355	355

download these results as csv

Task 1b Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

Simple Hit/Miss Count:

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
CSJ2	HAFR	-0.0394	0.2217	0.4828	FALSE
CSJ2	CSJ1	0.7058	0.9670	1.2281	TRUE
HAFR	CSJ1	0.4841	0.7453	1.0064	TRUE

download these results as csv

MRR Method:

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
CSJ2	HAFR	0.0442	0.3208	0.5974	TRUE
CSJ2	CSJ1	0.6904	0.9670	1.2436	TRUE
HAFR	CSJ1	0.3696	0.6462	0.9228	TRUE

download these results as csv

Task 1b Summary Results by Query Group

Simple Hit/Miss Count

	CSJ1	CSJ2	HAFR
1	0.55	1	0.82
2	0.6	0.8	0.8
3	0	0	1
4	0	1	1
5	0	0	0
6	0	0.83	0.83
7	1	1	0.83
8	0.33	0.33	0
9	0	0.8	1
10	0	1	1
11	0	1	0.5
12	1	1	1
13	0	0.6	0.6
14	0	0.5	0.5
15	0	0.5	0
16	1	0.5	1
17	0	0	0
18	0.5	1	0.5
19	0	1	1
20	0	0	1
21	0	0	0.5
22	0.44	0.89	0.89
23	1	1	1
24	0	1	1
25	0.5	1	1
26	0.64	0.91	0.91
27	0.5	1	0
28	0.5	1	0.5
29	1	1	1
30	0.75	1	0.75
31	1	1	1
32	0	1	0.86
33	0	1	1
34	0.5	0.5	0
35	0	1	1
36	0	0.8	0.6
37	0	0	0
38	0.5	1	1
39	0.43	1	1
40	1	1	0.5
41	0.13	0.88	0.63
42	0.67	0.92	0.5
43	0.67	0.67	0.56
44	0.56	1	1
45	1	1	0.67
46	0.5	0.75	0.75
47	0.75	1	1
48	1	1	1
49	1	1	1
50	1	1	1
51	0.5	0.75	1
52	0.25	1	0.75
53	0.33	1	0.83
54	0.67	1	1
55	0	1	0.75
56	0.5	0.5	0.5
57	1	1	0.4
58	0	1	1
59	0	1	1
60	1	1	0.5
61	0	1	1
62	0.33	1	1
63	1	1	1
64	0	1	0
65	0	0.67	1
66	0.5	0.75	1
67	0	1	0.67
68	1	1	1
69	0	1	1
70	1	1	1
71	0	0.67	1
72	1	1	1
73	1	1	0.5
74	1	1	1
75	0.33	0.67	0.67
76	0	1	0.5
77	1	1	1
78	1	1	1
79	0.5	1	1
80	0.33	1	1
81	0.33	0.5	1
82	0	1	1
83	0	1	1
84	0	0.25	0.5
85	0.5	1	0.75
86	1	1	1
87	0	1	0.33
88	0	1	1
89	0.43	0.86	1
90	0	1	1
91	0	1	0.5
92	0.5	1	1
93	0.5	1	1
94	0	1	1
95	0	1	0
96	1	0	1
97	0	1	1
98	0	1	1
99	0.75	1	1
100	0.33	1	1
101	0	0	0
102	0	1	1
103	0.38	0.75	0.63
104	0	1	1
105	1	0	1
106	0	0	0

download these results as csv

MRR Method

	CSJ1	CSJ2	HAFR
1	0.42	0.95	0.82
2	0.6	0.8	0.55
3	0	0	1
4	0	0.67	0.5
5	0	0	0
6	0	0.83	0.83
7	1	0.92	0.83
8	0.04	0.33	0
9	0	0.8	0.85
10	0	1	1
11	0	0.75	0.17
12	1	1	1
13	0	0.6	0.3
14	0	0.5	0.5
15	0	0.5	0
16	1	0.5	1
17	0	0	0
18	0.5	1	0.5
19	0	1	1
20	0	0	0.57
21	0	0	0.5
22	0.44	0.81	0.65
23	1	1	1
24	0	1	1
25	0.5	1	0.58
26	0.64	0.66	0.82
27	0.38	0.88	0
28	0.5	0.78	0.5
29	1	1	1
30	0.75	1	0.75
31	1	1	1
32	0	1	0.79
33	0	1	1
34	0.17	0.25	0
35	0	1	1
36	0	0.8	0.6
37	0	0	0
38	0.5	1	1
39	0.43	0.93	0.7
40	1	1	0.5
41	0.13	0.76	0.56
42	0.67	0.92	0.42
43	0.57	0.67	0.46
44	0.56	1	0.85
45	1	1	0.67
46	0.5	0.5	0.5
47	0.75	1	1
48	1	1	1
49	1	1	0.5
50	1	1	1
51	0.5	0.54	1
52	0.25	1	0.75
53	0.33	0.85	0.83
54	0.58	1	1
55	0	1	0.36
56	0.5	0.5	0.5
57	1	1	0.23
58	0	1	1
59	0	1	0.46
60	1	1	0.5
61	0	1	1
62	0.33	1	0.83
63	1	0.75	1
64	0	1	0
65	0	0.67	1
66	0.5	0.75	1
67	0	1	0.37
68	1	1	0.63
69	0	1	1
70	1	1	1
71	0	0.42	1
72	1	1	1
73	1	1	0.5
74	1	1	1
75	0.33	0.4	0.67
76	0	1	0.5
77	1	1	0.78
78	1	1	0.2
79	0.5	1	1
80	0.33	1	1
81	0.33	0.35	1
82	0	1	1
83	0	0.71	1
84	0	0.08	0.09
85	0.5	1	0.63
86	1	1	1
87	0	1	0.33
88	0	1	1
89	0.43	0.59	0.87
90	0	1	0.29
91	0	0.58	0.5
92	0.5	1	1
93	0.5	1	1
94	0	1	1
95	0	0.25	0
96	0.5	0	1
97	0	1	1
98	0	1	0.33
99	0.75	0.68	0.78
100	0.33	1	1
101	0	0	0
102	0	0.33	1
103	0.38	0.75	0.46
104	0	1	1
105	0.1	0	0.14
106	0	0	0

download these results as csv

Task 1b Summary Results by Query

Simple Hit/Miss Counting [3]

MRR Method [4]

Task 1c, IOACAS2's dataset Results

As argued by Jang, there are some abnormal in the IOACAS2 data set. Therefore, we just provide the raw result without any further analysis.

Task 1c Summary Results by Query Group

Simple Hit/Miss Count

	CSJ1	CSJ2	HAFR
1	0	1	0
2	0	0	1
3	0	1	1
4	0	0	0.33
5	0.27	0.27	0.36
6	0	0	0
7	0.5	0	0.5
8	0	0	0
9	0	0	1
10	1	1	1
11	0	0.5	0.75
12	0	1	1
13	0.14	0.14	0.43
14	0	0	0.4
15	0	0	0
16	1	1	1
17	0	0	1
18	0	1	0
19	0.5	0	0
20	0.25	0	0.5
21	0	0	1
22	0	0	0
23	0	0	0
24	0	0.14	0.14
25	0	0	0
26	0	0.25	1
27	0	1	0
28	0	0	0
29	0.5	0.38	0.63
30	0	0	1
31	0	0	0
32	0	0	1
33	0.33	0	0.33
34	0	1	1
35	0	1	0
36	0	0	1
37	0.11	0.33	0.89
38	0.5	0.5	0.5
39	0	1	1
40	0.5	0.5	1
41	0	1	1
42	0	0	1
43	0	0	1
44	0.5	1	1
45	0	0	0
46	0	0	1
47	0	1	1
48	0	1	0
49	0	0	0
50	0.13	0	0.88
51	0.25	0.5	0.75
52	0.33	0.33	0.67
53	0	0	0.5
54	0	0	0.5
55	0	0	1
56	0.33	0.33	0
57	0	0.33	1
58	0	0	0.29
59	1	1	0
60	0	1	1
61	0	0.5	0.5
62	0	0	0
63	0	0	0
64	0.44	0.33	0.33
65	0	0	0
66	0	0	0
67	0	0	0.5
68	0.6	0.7	0.5
69	0	0	0.33
70	0	0.6	0.3
71	0	0	1
72	0.5	0.33	0.33
73	0	0	0
74	0	0	0
75	0	0	1
76	0	0	0
77	1	1	1
78	0.83	0	0.17
79	0	0	1
80	0.54	0.62	0.85
81	0	0	1
82	0	0	0
83	0.43	0.14	1
84	0	0	0
85	0	0	1
86	0	1	0.25
87	1	1	1
88	0.25	0.75	0.75
89	0	1	1
90	0	0	0
91	0.5	0.5	1
92	0.5	1	1
93	0.4	0.6	0.8
94	0	0	0
95	0	0	1
96	0.33	0.33	1
97	0	0.5	1
98	1	0.5	0.5
99	0	0	0
100	0	0	1
101	0	1	1
102	0.5	0.5	0.5
103	0	1	0
104	1	0	1
105	0	1	0
106	0	0	0
107	0	1	1
108	0	1	1
109	1	0.5	1
110	0	0	1
111	0	0	0
112	0	0	0.5
113	0	0	0
114	1	1	1
115	0	0	0.5
116	0	0	0.5
117	0	0	0
118	1	0	1
119	0	0.2	0.4
120	0	0	1
121	0	0	0
122	1	0	1
123	0	1	1
124	0	0	0
125	0	1	1
126	0	0	0
127	0	0	0
128	0	0	0
129	0	1	1
130	0	0	0
131	1	1	1
132	0	0	0.5
133	1	1	0
134	1	1	1
135	0	0	0
136	0	1	1
137	0	1	1
138	0	1	1
139	1	1	1
140	1	0	1
141	1	1	1
142	0	0.5	1
143	0	0	1
144	0	0	1
145	0.33	0.33	1
146	0	0	0
147	0	1	0
148	0.5	1	0.5
149	1	1	1
150	0.5	0.5	1
151	0	1	1
152	0.33	0.33	1
153	0	0	1
154	0	1	0
155	0	0	1
156	1	0	1
157	1	1	1
158	0	0	0
159	0	0	0
160	0	0	0
161	0	0	0
162	1	1	0
163	0	0	1
164	0	1	0
165	0	1	1
166	0	0	0
167	1	1	1
168	1	1	0
169	1	1	1
170	1	1	1
171	0	0	1
172	1	1	1
173	1	0	1
174	1	0	1
175	0	1	1
176	0	1	1
177	0	0	1
178	1	0.5	1
179	1	0	1
180	0	0	1
181	1	1	1
182	0	0	1
183	0	1	1
184	0	1	0
185	0	1	1
186	0	0.33	0.33
187	0	1	1
188	0	0	1
189	0	1	1
190	0	0.5	1
191	1	1	1
192	0	1	0

download these results as csv

MRR Method

	CSJ1	CSJ2	HAFR
1	0	0.17	0
2	0	0	0.58
3	0	1	1
4	0	0	0.33
5	0.15	0.2	0.19
6	0	0	0
7	0.17	0	0.5
8	0	0	0
9	0	0	1
10	1	1	1
11	0	0.28	0.63
12	0	1	1
13	0.02	0.04	0.43
14	0	0	0.4
15	0	0	0
16	1	0.25	1
17	0	0	0.63
18	0	0.17	0
19	0.06	0	0
20	0.03	0	0.5
21	0	0	1
22	0	0	0
23	0	0	0
24	0	0.02	0.01
25	0	0	0
26	0	0.13	0.57
27	0	0.25	0
28	0	0	0
29	0.39	0.26	0.41
30	0	0	1
31	0	0	0
32	0	0	0.75
33	0.33	0	0.17
34	0	0.2	1
35	0	0.14	0
36	0	0	0.75
37	0.01	0.08	0.75
38	0.25	0.25	0.5
39	0	0.13	1
40	0.5	0.5	1
41	0	1	1
42	0	0	1
43	0	0	0.78
44	0.25	0.6	1
45	0	0	0
46	0	0	1
47	0	0.25	0.33
48	0	1	0
49	0	0	0
50	0.03	0	0.66
51	0.03	0.29	0.53
52	0.33	0.17	0.67
53	0	0	0.13
54	0	0	0.5
55	0	0	0.14
56	0.06	0.08	0
57	0	0.08	0.16
58	0	0	0.29
59	1	0.14	0
60	0	1	0.33
61	0	0.25	0.25
62	0	0	0
63	0	0	0
64	0.36	0.22	0.2
65	0	0	0
66	0	0	0
67	0	0	0.07
68	0.55	0.36	0.43
69	0	0	0.33
70	0	0.6	0.3
71	0	0	1
72	0.5	0.13	0.13
73	0	0	0
74	0	0	0
75	0	0	0.6
76	0	0	0
77	1	0.5	1
78	0.42	0	0.02
79	0	0	1
80	0.4	0.43	0.55
81	0	0	0.8
82	0	0	0
83	0.17	0.02	0.81
84	0	0	0
85	0	0	0.5
86	0	0.79	0.13
87	0.5	1	1
88	0.06	0.63	0.75
89	0	1	1
90	0	0	0
91	0.07	0.5	1
92	0.5	1	1
93	0.25	0.43	0.7
94	0	0	0
95	0	0	1
96	0.33	0.17	0.78
97	0	0.5	1
98	0.57	0.5	0.5
99	0	0	0
100	0	0	0.67
101	0	1	0.25
102	0.5	0.5	0.44
103	0	1	0
104	0.2	0	0.14
105	0	1	0
106	0	0	0
107	0	0.67	0.67
108	0	1	1
109	0.63	0.5	0.56
110	0	0	1
111	0	0	0
112	0	0	0.5
113	0	0	0
114	1	1	1
115	0	0	0.5
116	0	0	0.05
117	0	0	0
118	0.5	0	1
119	0	0.2	0.13
120	0	0	1
121	0	0	0
122	0.5	0	0.14
123	0	0.2	1
124	0	0	0
125	0	1	1
126	0	0	0
127	0	0	0
128	0	0	0
129	0	0.1	1
130	0	0	0
131	1	1	1
132	0	0	0.5
133	1	1	0
134	1	1	0.33
135	0	0	0
136	0	1	1
137	0	0.14	0.33
138	0	0.61	1
139	1	1	1
140	0.14	0	1
141	1	0.33	1
142	0	0.12	0.58
143	0	0	0.11
144	0	0	0.5
145	0.04	0.17	1
146	0	0	0
147	0	0.11	0
148	0.5	1	0.5
149	1	1	1
150	0.5	0.25	0.63
151	0	0.33	1
152	0.03	0.33	0.83
153	0	0	1
154	0	0.5	0
155	0	0	1
156	0.2	0	0.33
157	1	1	1
158	0	0	0
159	0	0	0
160	0	0	0
161	0	0	0
162	1	0.5	0
163	0	0	0.25
164	0	0.25	0
165	0	0.2	1
166	0	0	0
167	1	1	1
168	1	0.17	0
169	1	1	1
170	0.2	0.5	1
171	0	0	0.11
172	1	1	0.5
173	0.17	0	1
174	0.2	0	0.25
175	0	0.5	1
176	0	0.13	1
177	0	0	1
178	1	0.1	0.63
179	1	0	0.33
180	0	0	1
181	1	0.5	1
182	0	0	1
183	0	1	0.25
184	0	1	0
185	0	1	1
186	0	0.33	0.03
187	0	1	0.25
188	0	0	1
189	0	1	1
190	0	0.06	1
191	1	0.17	1
192	0	1	0

download these results as csv

Task 1c Summary Results by Query

Simple Hit/Miss Counting [5]

MRR Method [6]

Task 2 Results

Task 2 Overall Results

	CSJ1	CSJ2	HAFR
Simple Count	8.48	6.97	6.98
Total count	2040	2040	2020

download these results as csv

Task 2 Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
CSJ1	CSJ2	0.8116	1.2812	1.7509	TRUE
CSJ1	HAFR	0.8741	1.3438	1.8134	TRUE
CSJ2	HAFR	-0.4071	0.0625	0.5321	FALSE

download these results as csv

Task 2 Summary Results by Query Group

	CSJ1	CSJ2	HAFR
1	4.56	3.67	4.11
2	2	1.75	0.25
3	0	0	0
4	3.4	3.4	2.6
5	6.6	6.6	4.8
6	4	2.5	1.5
7	5.8	4.4	2.2
8	5.4	6.2	4
9	2	1.5	2.5
10	4.33	2.56	3
11	8.3	7.56	6.33
12	9.25	7.54	6.93
13	9.22	7.57	7.35
14	8.63	7.79	8.69
15	4.36	3.93	3.93
16	8.43	6.35	7.38
17	8.67	7.15	5.41
18	8.39	6.03	8.14
19	9.22	7.94	7.69
20	9.1	8.12	8.27
21	8.91	6	3.26
22	9.17	7.82	8.1
23	8.29	6.87	4.93
24	8.9	7.61	6.86
25	8.1	7.39	7.45
26	7.52	4.86	6
27	7.93	6.42	4.76
28	8.98	7.93	7.07
29	9.04	8.03	7.31
30	9.42	8.03	9.05
31	9.57	8.7	7.83
32	9.59	8.08	7.84
33	7.59	6.1	6.52
34	9.29	7.51	8.38
35	8.74	6.43	7.5
36	7.71	5.79	4.33
37	7.43	5.21	6.25
38	8.28	7.11	7.7
39	6.97	5.91	7.19
40	8.81	6.96	6.44
41	9.15	7.14	7.59
42	6.86	6.12	5.71
43	8.2	5.88	6.68
44	9.19	8.11	6.56
45	7.11	5.13	5.09
46	7.66	5.42	6.7
47	7.92	5.69	3.38
48	8.18	6	7.59

download these results as csv

Task 2 Summary Results by Query

[7]

Runtime Results

Participant	Task	Runtime (min)	Machine
CSJ1	1a	~55	BIGWIN
CSJ1	1b	7	BIGWIN
CSJ1	2	~740	BIGWIN
CSJ2	1a	~1800	BIGWIN
CSJ2	1b	~210	BIGWIN
CSJ2	2	~730	BIGWIN
HAFR	1a	1689	BEER 2
HAFR	1b	247	BEER 2
HAFR	2	242	BEER 2

download these results as csv