Difference between revisions of "2010:Query-by-Tapping Results"

From MIREX Wiki
(qbt, result)
 
 
(3 intermediate revisions by one other user not shown)
Line 15: Line 15:
 
====Team ID====
 
====Team ID====
 
{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
 
{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
−
|- style="background: #efefef;"
+
|- style="background: yellow;"
 
! width="80" | Sub code  
 
! width="80" | Sub code  
 
! width="200" | Submission name  
 
! width="200" | Submission name  
Line 24: Line 24:
 
| QBT simbals ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2010/HRFA1.pdf PDF] || [http://dept-info.labri.fr/~hanna Pierre Hanna], [http://simbals.labri.fr Matthias Robine], [http://simbals.labri.fr Pascal Ferraro], [http://simbals.labri.fr Julien Allali]
 
| QBT simbals ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2010/HRFA1.pdf PDF] || [http://dept-info.labri.fr/~hanna Pierre Hanna], [http://simbals.labri.fr Matthias Robine], [http://simbals.labri.fr Pascal Ferraro], [http://simbals.labri.fr Julien Allali]
 
|-
 
|-
−
! ML4
+
! ML
 
| QBT-M&O ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2010/ML4.pdf PDF] || [http://www-sipl.technion.ac.il/ Merabi Mirel], [http://www-sipl.technion.ac.il/ Oron Levy]
 
| QBT-M&O ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2010/ML4.pdf PDF] || [http://www-sipl.technion.ac.il/ Merabi Mirel], [http://www-sipl.technion.ac.il/ Oron Levy]
 
|}
 
|}
 +
 
===Task 1 Results===
 
===Task 1 Results===
  
Line 40: Line 41:
  
 
Simple Hit/Miss Count:
 
Simple Hit/Miss Count:
−
<csv>2010/qbt/friedman/QbtTask1aSimple_friedman_tukeyKramerHSD.csv</csv>
+
<csv>2010/qbt/friedman/QbtTask1aSimpleByGroup.friedman.tukeyKramerHSD.csv</csv>
  
−
[[Image:2010_sqbttask1asimple_friedman_mean_ranks.png]]
+
[[Image:2010QbtTask1aSimpleByGroup.friedman.tukeyKramerHSD.png|500px]]
  
 
MRR Method:
 
MRR Method:
−
<csv>2010/qbt/friedman/Qbt1aTask2Mrr_friedman_tukeyKramerHSD.csv</csv>
+
<csv>2010/qbt/friedman/QbtTask1aMrrByGroup.friedman.tukeyKramerHSD.csv</csv>
  
−
[[Image:2010_sqbt1atask2mrr_friedman_mean_ranks.png]]
+
[[Image:2010QbtTask1aMrrByGroup.friedman.tukeyKramerHSD.png|500px]]
  
 
====Task 1a Summary Results by Query Group====
 
====Task 1a Summary Results by Query Group====
Line 74: Line 75:
  
 
Simple Hit/Miss Count:
 
Simple Hit/Miss Count:
−
<csv>2010/qbt/friedman/QbtTask1bSimple_friedman_tukeyKramerHSD.csv</csv>
+
<csv>2010/qbt/friedman/QbtTask1bSimpleByGroup.friedman.tukeyKramerHSD.csv</csv>
  
−
[[Image:2010_sqbttask1bsimple_friedman_mean_ranks.png]]
+
[[Image:2010QbtTask1bSimpleByGroup.friedman.tukeyKramerHSD.png|500px]]
  
 
MRR Method:
 
MRR Method:
−
<csv>2010/qbt/friedman/QbtTask1bMrr_friedman_tukeyKramerHSD.csv</csv>
+
<csv>2010/qbt/friedman/QbtTask1bMrrByGroup.friedman.tukeyKramerHSD.csv</csv>
  
−
[[Image:2010_sqbttask1bmrr_friedman_mean_ranks.png ]]
+
[[Image:2010QbtTask1bMrrByGroup.friedman.tukeyKramerHSD.png|500px ]]
  
 
====Task 1b Summary Results by Query Group====
 
====Task 1b Summary Results by Query Group====
Line 108: Line 109:
  
 
Simple Hit/Miss Count:
 
Simple Hit/Miss Count:
−
<csv>2010/qbt/friedman/QbtTask2Simple_friedman_tukeyKramerHSD.csv</csv>
+
<csv>2010/qbt/friedman/QbtTask2SimpleByGroup.friedman.tukeyKramerHSD.csv</csv>
  
−
[[Image:2010_sqbttask2simple_friedman_mean_ranks.png]]
+
[[Image:2010QbtTask2SimpleByGroup.friedman.tukeyKramerHSD.png|500px]]
  
 
MRR Method:
 
MRR Method:
−
<csv>2010/qbt/friedman/QbtTask2Mrr_friedman_tukeyKramerHSD.csv</csv>
+
<csv>2010/qbt/friedman/QbtTask2MrrByGroup.friedman.tukeyKramerHSD.csv</csv>
  
−
[[Image:2010_sqbttask2mrr_friedman_mean_ranks.png]]
+
[[Image:2010QbtTask2MrrByGroup.friedman.tukeyKramerHSD.png|500px]]
  
 
====Task 2 Summary Results by Query Group====
 
====Task 2 Summary Results by Query Group====

Latest revision as of 14:47, 3 August 2010

Introduction

These are the results for the 2010 running of the Query-by-tappingn task. For background information about this task set please refer to the 2010:Query_by_Tapping page.

Task Descriptions

Task 1 Goto Task 1 Results: The first subtask is the same as last year. In this subtask, submitted systems take a symbolic sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth, as well as the simple hit(1)/miss(0) counting, is calculated over the top 10 returns. Two data sets are used:

  • Jang's Dataset Roger Jang's MIR-QBT: This dataset contains both wav files (recorded via microphone) and onset files (human-labeled onset time). 136 ground truth songs with 890 queries.
  • Hsiao's Dataset Show Hsiao's QBT_symbolic: This dataset contains only onset files (obtained from the user's tapping on keyboard). 143 ground truth songs with 410 queries.


Task 2 Goto Task 2 Results:The second subtask is the same as last year too. In this subtask, submitted systems take a wave-file sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth, as well as the simple hit(1)/miss(0) counting, is calculated over the top 10 returns. Only Roger Jang's data http://neural.cs.nthu.edu.tw/jang2/dataSet/qbt4public/MIR-QBT.rar MIR-QBT] is used as the other dataset has no wave files.

General Legend

Team ID

Sub code Submission name Abstract Contributors
HRFA1 QBT simbals PDF Pierre Hanna, Matthias Robine, Pascal Ferraro, Julien Allali
ML QBT-M&O PDF Merabi Mirel, Oron Levy

Task 1 Results

Task 1a, Jang's Dataset Results

Task 1a Overall Results
HAFR ML
Simple Count 0.875 0.876
MRR 0.769 0.798
Total Count 890 890

download these results as csv

Task 1a Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

Simple Hit/Miss Count:

TeamID TeamID Lowerbound Mean Upperbound Significance
HAFR ML -0.0641 0.0368 0.1376 FALSE

download these results as csv

2010QbtTask1aSimpleByGroup.friedman.tukeyKramerHSD.png

MRR Method:

TeamID TeamID Lowerbound Mean Upperbound Significance
ML HAFR -0.0260 0.1029 0.2318 FALSE

download these results as csv

2010QbtTask1aMrrByGroup.friedman.tukeyKramerHSD.png

Task 1a Summary Results by Query Group

Simple Hit/Miss Counting

HAFR ML
1 0.92 1.00
2 1.00 1.00
3 1.00 1.00
4 0.00 1.00
5 1.00 1.00
6 1.00 0.75
7 1.00 1.00
8 1.00 1.00
9 1.00 1.00
10 1.00 1.00
11 1.00 0.50
12 1.00 1.00
13 1.00 1.00
14 1.00 1.00
15 1.00 1.00
16 1.00 1.00
17 0.93 0.93
18 0.85 0.77
19 0.50 0.38
20 1.00 1.00
21 1.00 0.50
22 0.50 1.00
23 0.00 0.00
24 1.00 1.00
25 1.00 1.00
26 0.00 0.00
27 0.94 1.00
28 1.00 1.00
29 1.00 1.00
30 0.75 0.67
31 1.00 1.00
32 1.00 1.00
33 0.74 0.88
34 0.92 0.83
35 1.00 1.00
36 1.00 1.00
37 1.00 0.00
38 1.00 1.00
39 1.00 1.00
40 1.00 1.00
41 0.75 0.83
42 1.00 1.00
43 1.00 0.90
44 0.75 0.75
45 1.00 1.00
46 1.00 1.00
47 1.00 0.50
48 1.00 0.67
49 1.00 1.00
50 1.00 1.00
51 0.80 0.40
52 1.00 1.00
53 0.60 0.60
54 1.00 1.00
55 0.98 1.00
56 1.00 1.00
57 0.80 1.00
58 1.00 1.00
59 0.60 0.20
60 1.00 0.57
61 1.00 1.00
62 1.00 1.00
63 1.00 1.00
64 0.50 0.75
65 1.00 1.00
66 1.00 1.00
67 1.00 1.00
68 0.78 0.78
69 0.82 0.85
70 1.00 1.00
71 0.00 1.00
72 0.92 0.83
73 0.86 0.93
74 1.00 1.00
75 1.00 0.83
76 1.00 1.00
77 0.00 0.00
78 0.00 1.00
79 0.50 1.00
80 1.00 1.00
81 1.00 1.00
82 1.00 1.00
83 1.00 1.00
84 1.00 1.00
85 0.86 0.71
86 1.00 1.00
87 0.75 0.67
88 1.00 1.00
89 0.67 1.00
90 1.00 0.50
91 1.00 1.00
92 0.25 0.25
93 1.00 1.00
94 0.67 0.83
95 1.00 1.00
96 1.00 1.00
97 1.00 1.00
98 0.50 1.00
99 0.88 0.88
100 1.00 1.00
101 1.00 0.83
102 1.00 0.88
103 1.00 1.00
104 1.00 1.00
105 1.00 0.96
106 0.68 0.77
107 0.94 0.94
108 0.56 0.75
109 0.81 0.88
110 1.00 1.00
111 1.00 0.78
112 1.00 1.00
113 1.00 1.00
114 1.00 1.00
115 0.60 0.60
116 0.75 0.50
117 1.00 1.00
118 1.00 1.00
119 0.67 0.67
120 0.67 0.67
121 0.67 0.33
122 0.67 0.33
123 0.33 0.67
124 0.33 0.33
125 0.33 1.00
126 0.67 0.67
127 1.00 1.00
128 0.67 0.67
129 0.50 0.50
130 0.50 0.50
131 1.00 1.00
132 1.00 1.00
133 1.00 0.00
134 1.00 1.00
135 1.00 1.00
136 1.00 1.00

download these results as csv

MRR Method

HAFR ML
1 0.89 1.00
2 1.00 1.00
3 1.00 1.00
4 0.00 1.00
5 1.00 1.00
6 0.88 0.75
7 1.00 1.00
8 0.75 1.00
9 0.25 1.00
10 1.00 1.00
11 0.60 0.50
12 1.00 1.00
13 1.00 1.00
14 0.94 1.00
15 1.00 1.00
16 0.81 0.80
17 0.93 0.79
18 0.64 0.72
19 0.30 0.31
20 1.00 1.00
21 1.00 0.50
22 0.50 0.32
23 0.00 0.00
24 0.50 0.50
25 0.91 1.00
26 0.00 0.00
27 0.76 0.91
28 1.00 1.00
29 1.00 1.00
30 0.75 0.58
31 0.33 0.20
32 1.00 0.83
33 0.72 0.72
34 0.92 0.83
35 1.00 1.00
36 0.78 0.81
37 1.00 0.00
38 1.00 1.00
39 1.00 1.00
40 0.96 1.00
41 0.58 0.46
42 1.00 1.00
43 0.88 0.83
44 0.62 0.75
45 0.42 0.42
46 0.71 1.00
47 0.57 0.50
48 1.00 0.42
49 1.00 1.00
50 0.44 0.78
51 0.63 0.30
52 1.00 1.00
53 0.60 0.37
54 0.50 1.00
55 0.96 1.00
56 1.00 1.00
57 0.80 0.90
58 1.00 1.00
59 0.42 0.05
60 0.59 0.57
61 1.00 1.00
62 0.96 0.96
63 0.11 0.12
64 0.28 0.62
65 1.00 1.00
66 1.00 1.00
67 1.00 1.00
68 0.55 0.69
69 0.67 0.77
70 0.78 0.79
71 0.00 1.00
72 0.82 0.70
73 0.82 0.83
74 0.80 1.00
75 0.85 0.83
76 1.00 0.92
77 0.00 0.00
78 0.00 1.00
79 0.38 0.67
80 1.00 1.00
81 1.00 1.00
82 1.00 1.00
83 0.88 1.00
84 0.50 1.00
85 0.16 0.71
86 1.00 1.00
87 0.69 0.49
88 1.00 1.00
89 0.42 1.00
90 1.00 0.25
91 1.00 1.00
92 0.12 0.25
93 1.00 1.00
94 0.29 0.62
95 0.94 0.83
96 1.00 1.00
97 1.00 1.00
98 0.50 0.55
99 0.79 0.88
100 1.00 1.00
101 0.57 0.83
102 1.00 0.88
103 1.00 1.00
104 1.00 0.98
105 0.94 0.94
106 0.59 0.71
107 0.68 0.82
108 0.46 0.48
109 0.65 0.88
110 0.97 1.00
111 0.82 0.78
112 0.94 1.00
113 1.00 1.00
114 0.80 0.73
115 0.31 0.50
116 0.54 0.38
117 1.00 1.00
118 1.00 1.00
119 0.67 0.67
120 0.67 0.67
121 0.40 0.33
122 0.50 0.04
123 0.33 0.67
124 0.33 0.33
125 0.17 0.44
126 0.67 0.67
127 1.00 1.00
128 0.67 0.67
129 0.25 0.25
130 0.50 0.50
131 1.00 1.00
132 1.00 1.00
133 0.14 0.00
134 1.00 1.00
135 1.00 1.00
136 1.00 1.00

download these results as csv

Task 1a Summary Results by Query

Simple Hit/Miss Counting [1]

MRR Method [2]


Task 1b, Hsiao's Dataset Results

Task 1b Overall Results
HAFR ML
Simple Count 0.88 0.85
MRR 0.70 0.71
Total Count 890.00 890.00

download these results as csv

Task 1b Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

Simple Hit/Miss Count:

TeamID TeamID Lowerbound Mean Upperbound Significance
HAFR ML -0.0419 0.0280 0.0979 FALSE

download these results as csv

2010QbtTask1bSimpleByGroup.friedman.tukeyKramerHSD.png

MRR Method:

TeamID TeamID Lowerbound Mean Upperbound Significance
ML HAFR -0.1270 -0.0140 0.0990 FALSE

download these results as csv

2010QbtTask1bMrrByGroup.friedman.tukeyKramerHSD.png

Task 1b Summary Results by Query Group

Simple Hit/Miss Counting

HAFR ML
1 1.00 1.00
2 NaN NaN
3 1.00 1.00
4 1.00 1.00
5 1.00 1.00
6 1.00 1.00
7 1.00 1.00
8 1.00 1.00
9 1.00 1.00
10 0.67 1.00
11 1.00 1.00
12 1.00 0.75
13 1.00 1.00
14 1.00 1.00
15 NaN NaN
16 1.00 1.00
17 1.00 1.00
18 1.00 1.00
19 1.00 1.00
20 1.00 1.00
21 1.00 1.00
22 1.00 1.00
23 0.83 1.00
24 0.67 0.67
25 1.00 0.00
26 0.00 0.00
27 1.00 1.00
28 NaN NaN
29 1.00 1.00
30 1.00 1.00
31 NaN NaN
32 1.00 1.00
33 1.00 1.00
34 1.00 1.00
35 NaN NaN
36 0.00 0.00
37 0.00 0.50
38 0.86 1.00
39 1.00 1.00
40 0.50 0.00
41 1.00 1.00
42 1.00 0.33
43 1.00 1.00
44 1.00 1.00
45 1.00 1.00
46 1.00 1.00
47 1.00 1.00
48 1.00 1.00
49 0.67 0.67
50 1.00 1.00
51 1.00 0.00
52 1.00 1.00
53 0.89 0.89
54 1.00 1.00
55 1.00 0.00
56 0.50 1.00
57 1.00 1.00
58 NaN NaN
59 1.00 1.00
60 1.00 1.00
61 0.75 0.75
62 0.50 1.00
63 1.00 1.00
64 NaN NaN
65 1.00 1.00
66 NaN NaN
67 1.00 1.00
68 NaN NaN
69 1.00 1.00
70 1.00 0.20
71 NaN NaN
72 1.00 1.00
73 1.00 1.00
74 NaN NaN
75 1.00 1.00
76 1.00 0.60
77 NaN NaN
78 1.00 1.00
79 1.00 1.00
80 1.00 1.00
81 1.00 0.00
82 1.00 0.00
83 1.00 1.00
84 1.00 1.00
85 0.00 0.00
86 1.00 1.00
87 1.00 0.75
88 0.60 1.00
89 1.00 1.00
90 1.00 1.00
91 1.00 1.00
92 1.00 1.00
93 1.00 1.00
94 0.75 1.00
95 0.67 0.67
96 1.00 1.00
97 0.00 0.00
98 1.00 1.00
99 1.00 1.00
100 1.00 1.00
101 1.00 1.00
102 1.00 1.00
103 NaN NaN
104 1.00 1.00
105 1.00 1.00
106 0.33 0.00
107 NaN NaN
108 1.00 1.00
109 1.00 1.00
110 1.00 1.00
111 1.00 1.00
112 1.00 1.00
113 1.00 1.00
114 0.50 1.00
115 NaN NaN
116 0.00 0.00
117 1.00 1.00
118 1.00 0.67
119 0.00 0.00
120 0.67 1.00
121 0.67 1.00
122 1.00 1.00
123 1.00 1.00
124 1.00 1.00
125 0.00 0.00
126 1.00 1.00
127 1.00 1.00
128 1.00 1.00
129 1.00 1.00
130 0.00 0.00
131 0.00 0.00
132 1.00 1.00
133 1.00 0.00
134 0.50 0.50
135 1.00 1.00
136 1.00 1.00
137 1.00 1.00
138 1.00 1.00
139 1.00 0.80
140 0.00 0.00
141 1.00 1.00
142 1.00 1.00
143 1.00 1.00

download these results as csv

MRR Method

HAFR ML
1 1.00 1.00
2 NaN NaN
3 0.83 0.88
4 0.75 0.75
5 1.00 1.00
6 0.50 0.50
7 1.00 1.00
8 1.00 1.00
9 1.00 1.00
10 0.50 0.61
11 1.00 1.00
12 0.81 0.75
13 1.00 1.00
14 0.42 0.75
15 NaN NaN
16 1.00 1.00
17 1.00 1.00
18 1.00 1.00
19 1.00 1.00
20 1.00 1.00
21 1.00 1.00
22 1.00 1.00
23 0.47 1.00
24 0.67 0.67
25 0.75 0.00
26 0.00 0.00
27 0.67 0.50
28 NaN NaN
29 1.00 1.00
30 1.00 1.00
31 NaN NaN
32 0.50 1.00
33 0.56 0.50
34 0.61 0.67
35 NaN NaN
36 0.00 0.00
37 0.00 0.10
38 0.26 0.93
39 1.00 1.00
40 0.07 0.00
41 1.00 1.00
42 0.83 0.17
43 0.75 0.46
44 0.42 0.75
45 0.56 0.69
46 1.00 1.00
47 1.00 1.00
48 1.00 1.00
49 0.14 0.21
50 1.00 1.00
51 0.12 0.00
52 1.00 1.00
53 0.83 0.89
54 0.37 0.23
55 1.00 0.00
56 0.25 1.00
57 0.90 1.00
58 NaN NaN
59 0.50 0.50
60 0.62 0.62
61 0.75 0.62
62 0.50 0.31
63 0.67 0.83
64 NaN NaN
65 0.49 0.46
66 NaN NaN
67 0.20 0.75
68 NaN NaN
69 0.85 0.69
70 0.41 0.20
71 NaN NaN
72 1.00 0.75
73 1.00 1.00
74 NaN NaN
75 1.00 1.00
76 0.85 0.13
77 NaN NaN
78 0.83 0.67
79 0.58 0.75
80 0.33 1.00
81 0.35 0.00
82 0.50 0.00
83 1.00 1.00
84 0.62 1.00
85 0.00 0.00
86 1.00 1.00
87 1.00 0.62
88 0.43 1.00
89 0.69 0.85
90 0.78 0.50
91 1.00 0.88
92 1.00 1.00
93 1.00 1.00
94 0.16 0.33
95 0.67 0.67
96 1.00 1.00
97 0.00 0.00
98 0.75 1.00
99 1.00 1.00
100 1.00 1.00
101 1.00 0.87
102 1.00 1.00
103 NaN NaN
104 1.00 1.00
105 0.33 1.00
106 0.11 0.00
107 NaN NaN
108 1.00 1.00
109 0.33 0.33
110 0.10 0.50
111 0.50 0.33
112 0.78 0.73
113 1.00 1.00
114 0.17 0.56
115 NaN NaN
116 0.00 0.00
117 0.75 0.75
118 0.67 0.33
119 0.00 0.00
120 0.50 0.58
121 0.67 0.70
122 0.88 0.67
123 0.78 0.49
124 0.70 0.60
125 0.00 0.00
126 1.00 1.00
127 0.88 0.62
128 0.78 1.00
129 1.00 1.00
130 0.00 0.00
131 0.00 0.00
132 0.75 1.00
133 0.48 0.00
134 0.06 0.17
135 1.00 1.00
136 1.00 1.00
137 1.00 1.00
138 0.38 0.33
139 0.44 0.43
140 0.00 0.00
141 0.69 1.00
142 0.50 0.67
143 1.00 1.00

download these results as csv

Task 1b Summary Results by Query

Simple Hit/Miss Counting [3]

MRR Method [4]

Task 2 Results

Task 2 Overall Results
HAFR ML
Simple Count 0.787 0.872
MRR 0.664 0.781
Total Count 890 890

download these results as csv

Task 2 Friedman's Test for Significant Differences

The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

Simple Hit/Miss Count:

TeamID TeamID Lowerbound Mean Upperbound Significance
ML HAFR 0.0162 0.1250 0.2338 TRUE

download these results as csv

2010QbtTask2SimpleByGroup.friedman.tukeyKramerHSD.png

MRR Method:

TeamID TeamID Lowerbound Mean Upperbound Significance
ML HAFR 0.2316 0.3750 0.5184 TRUE

download these results as csv

2010QbtTask2MrrByGroup.friedman.tukeyKramerHSD.png

Task 2 Summary Results by Query Group

Simple Hit/Miss Counting

HAFR ML
1 0.69 1.00
2 1.00 1.00
3 1.00 1.00
4 0.00 1.00
5 1.00 1.00
6 1.00 0.75
7 1.00 1.00
8 1.00 1.00
9 1.00 1.00
10 1.00 1.00
11 1.00 1.00
12 1.00 1.00
13 1.00 1.00
14 1.00 1.00
15 1.00 1.00
16 0.90 1.00
17 0.93 0.87
18 0.61 0.77
19 0.75 0.38
20 1.00 1.00
21 1.00 0.50
22 1.00 1.00
23 0.00 0.00
24 1.00 1.00
25 1.00 1.00
26 0.00 0.00
27 0.71 0.97
28 0.67 1.00
29 1.00 1.00
30 0.67 0.67
31 0.00 1.00
32 1.00 1.00
33 0.59 0.86
34 0.92 0.83
35 1.00 1.00
36 1.00 1.00
37 1.00 0.00
38 1.00 1.00
39 1.00 1.00
40 0.86 0.93
41 0.83 0.92
42 1.00 1.00
43 0.90 0.90
44 0.75 0.75
45 1.00 1.00
46 0.83 1.00
47 1.00 0.50
48 1.00 0.67
49 1.00 1.00
50 1.00 1.00
51 0.80 0.40
52 1.00 0.50
53 0.20 0.60
54 1.00 1.00
55 0.84 1.00
56 1.00 1.00
57 0.80 1.00
58 1.00 1.00
59 0.40 0.20
60 0.57 0.57
61 0.82 1.00
62 0.75 0.92
63 0.00 1.00
64 0.50 0.75
65 1.00 1.00
66 1.00 1.00
67 1.00 1.00
68 0.56 0.78
69 0.64 0.85
70 0.75 1.00
71 0.00 1.00
72 0.92 0.83
73 0.71 1.00
74 1.00 1.00
75 0.83 0.83
76 1.00 1.00
77 0.00 0.00
78 0.00 1.00
79 0.75 0.75
80 1.00 1.00
81 1.00 1.00
82 1.00 1.00
83 1.00 1.00
84 1.00 1.00
85 0.43 0.86
86 1.00 1.00
87 0.67 0.58
88 1.00 1.00
89 1.00 1.00
90 1.00 0.50
91 1.00 1.00
92 0.25 0.25
93 1.00 1.00
94 0.83 0.83
95 0.88 1.00
96 1.00 1.00
97 1.00 1.00
98 0.50 1.00
99 0.88 0.88
100 1.00 1.00
101 1.00 1.00
102 1.00 0.88
103 1.00 1.00
104 0.89 0.96
105 0.81 0.96
106 0.73 0.82
107 0.72 0.89
108 0.44 0.75
109 0.81 0.94
110 0.93 0.93
111 1.00 1.00
112 1.00 1.00
113 0.71 0.86
114 0.60 1.00
115 0.60 0.40
116 0.75 0.25
117 1.00 1.00
118 1.00 1.00
119 0.67 0.67
120 0.67 0.67
121 1.00 0.67
122 0.67 0.00
123 0.67 0.67
124 0.33 0.33
125 1.00 1.00
126 1.00 1.00
127 1.00 1.00
128 0.67 0.33
129 0.50 0.50
130 0.50 1.00
131 1.00 1.00
132 1.00 1.00
133 0.00 0.00
134 1.00 1.00
135 1.00 1.00
136 0.00 1.00

download these results as csv

MRR Method

HAFR ML
1 0.69 0.96
2 1.00 1.00
3 1.00 1.00
4 0.00 1.00
5 0.67 1.00
6 0.83 0.75
7 1.00 1.00
8 1.00 0.58
9 0.10 1.00
10 1.00 1.00
11 0.67 1.00
12 1.00 1.00
13 1.00 1.00
14 0.89 1.00
15 1.00 1.00
16 0.70 0.78
17 0.79 0.78
18 0.34 0.67
19 0.33 0.31
20 1.00 1.00
21 1.00 0.50
22 0.55 0.32
23 0.00 0.00
24 0.50 0.50
25 0.93 1.00
26 0.00 0.00
27 0.64 0.91
28 0.37 1.00
29 1.00 1.00
30 0.53 0.58
31 0.00 0.20
32 1.00 0.83
33 0.56 0.68
34 0.92 0.83
35 1.00 1.00
36 0.58 0.81
37 1.00 0.00
38 1.00 1.00
39 1.00 1.00
40 0.82 0.93
41 0.47 0.49
42 1.00 1.00
43 0.78 0.83
44 0.75 0.75
45 0.33 0.42
46 0.62 1.00
47 1.00 0.50
48 0.43 0.42
49 1.00 0.75
50 0.51 0.73
51 0.70 0.30
52 1.00 0.50
53 0.20 0.50
54 0.33 1.00
55 0.75 0.96
56 1.00 1.00
57 0.80 0.90
58 0.91 1.00
59 0.09 0.05
60 0.41 0.57
61 0.82 1.00
62 0.69 0.92
63 0.00 0.14
64 0.50 0.62
65 1.00 1.00
66 1.00 1.00
67 0.25 1.00
68 0.44 0.66
69 0.49 0.73
70 0.54 0.80
71 0.00 1.00
72 0.80 0.72
73 0.60 0.95
74 0.90 1.00
75 0.83 0.83
76 1.00 1.00
77 0.00 0.00
78 0.00 1.00
79 0.38 0.62
80 1.00 1.00
81 1.00 1.00
82 1.00 1.00
83 0.88 1.00
84 0.50 1.00
85 0.22 0.79
86 1.00 1.00
87 0.50 0.44
88 0.75 1.00
89 0.58 1.00
90 0.56 0.25
91 0.87 1.00
92 0.06 0.25
93 1.00 1.00
94 0.49 0.62
95 0.70 0.75
96 0.50 1.00
97 0.79 1.00
98 0.50 0.55
99 0.79 0.88
100 1.00 1.00
101 0.52 0.85
102 1.00 0.88
103 0.89 1.00
104 0.79 0.89
105 0.81 0.94
106 0.58 0.71
107 0.67 0.77
108 0.33 0.47
109 0.62 0.82
110 0.90 0.93
111 0.81 1.00
112 1.00 1.00
113 0.71 0.86
114 0.44 0.73
115 0.35 0.40
116 0.58 0.25
117 0.75 1.00
118 1.00 1.00
119 0.50 0.67
120 0.67 0.67
121 0.83 0.67
122 0.25 0.00
123 0.44 0.67
124 0.17 0.33
125 0.28 0.39
126 1.00 1.00
127 1.00 1.00
128 0.67 0.08
129 0.05 0.25
130 0.50 0.57
131 1.00 0.67
132 0.60 1.00
133 0.00 0.00
134 1.00 1.00
135 1.00 1.00
136 0.00 1.00

download these results as csv

Task 2 Summary Results by Query

Simple Hit/Miss Counting [5]

MRR Method [6]

Runtime Results

Participant Task Runtime (min) Machine
HRFA task1a 13 nema-c-1-2
HRFA task1b 5 nema-c-1-2
HRFA task2 37 nema-c-1-2
ML task1a 10 1.83 GHz. 1Gb RAM
ML task1b 4 2 GHz dual core. 2Gb RAM
ML task2 13 1.83 GHz. 1Gb RAM

download these results as csv

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2010:Query-by-Tapping_Results&oldid=7678"