2006:QBSH: Query-by-Singing/Humming Results - Revision history

Kahyun Choi: /* Introduction */

2011-12-20T04:41:04Z

‎Introduction

Singh14: /* Team ID */

2010-07-31T21:06:50Z

‎Team ID

Singh14: /* Team ID */

2010-07-31T21:03:32Z

‎Team ID

IMIRSELBot: Robot: Automated text replacement (-http://www.music-ir.org/evaluation/MIREX/2006_abstracts/ +http://www.music-ir.org/mirex/abstracts/2006/)

2010-05-14T01:32:39Z

Robot: Automated text replacement (-http://www.music-ir.org/evaluation/MIREX/2006_abstracts/ +http://www.music-ir.org/mirex/abstracts/2006/)

IMIRSELBot: Robot: Automated text replacement (-]*)> +2006/)

2010-05-14T00:37:29Z

Robot: Automated text replacement (-<csv([^>]*)> +<csv\1>2006/)

IMIRSELBot: Robot: Automated text replacement (-\[\[([A-Z][^:]+)\]\] +2006:\1)

2010-05-13T18:05:24Z

Robot: Automated text replacement (-\[\[([A-Z][^:]+)\]\] +2006:\1)

Jhlee at 01:13, 3 October 2006

2006-10-03T01:13:41Z

Jhlee at 01:09, 3 October 2006

2006-10-03T01:09:31Z

Mertbay at 20:25, 2 October 2006

2006-10-02T20:25:04Z

Mertbay: /* Introduction */

2006-10-02T20:22:21Z

‎Introduction

@@ Line 1: / Line 1: @@
 [[Category: Results]]
 ==Introduction==
-These are the results for the 2006 running of the QBSH: Query-by-Singing Humming task set. For background information about this task set please refer to the [[QBSH: Query-by-Singing/Humming]] page.
+These are the results for the 2006 running of the QBSH: Query-by-Singing Humming task set. For background information about this task set please refer to the [[2006:QBSH: Query-by-Singing/Humming]] page.
 QBSH task consists of two subtasks. The first subtask is known as '''Known-Item Retrieval'''. In this subtask, submitted systems take a sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth is calculated over the top 20 returns. The test database consists of 48 ground-truth MIDIs + 2000 Essen Collection MIDI noise files. The query database consists of 2797 sung queries.

@@ Line 15: / Line 15: @@
 '''NM''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_mikkila.pdf Kjell Lemström, Niko Mikkilä, Veli Mäkinen and Esko Ukkonen]<br />
 '''RJ''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_jang.pdf Jyh-Shing Roger Jang and Nien-Jung Lee]<br />
-'''RL''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_tararira.pdf Ernesto Lopez and Mart├¡n Rocamora]<br />
+'''RL''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_tararira.pdf Ernesto López and Martín Rocamora]<br />
 '''RT''' = [https://www.music-ir.org/mirex/abstracts/2006/SMS_QBSH_typke.pdf Rainer Typke, Frans Wiering and Remco C. Veltkamp]<br />
 '''XW''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_wu.pdf Xiao Wu and Ming Li]<br />

@@ Line 13: / Line 13: @@
 '''CS3''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_sailer.pdf Christian Sailer-warp]<br />
 '''FH''' = Pascal Ferraro and Pierre Hanna<br />
-'''NM''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_mikkila.pdf Kjell Lemstr├╢m, Niko Mikkil├ñ, Veli M├ñkinen and Esko Ukkonen]<br />
+'''NM''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_mikkila.pdf Kjell Lemström, Niko Mikkilä, Veli Mäkinen and Esko Ukkonen]<br />
 '''RJ''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_jang.pdf Jyh-Shing Roger Jang and Nien-Jung Lee]<br />
 '''RL''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_tararira.pdf Ernesto Lopez and Mart├¡n Rocamora]<br />

@@ Line 8: / Line 8: @@
 ===General Legend===
 ====Team ID====
-'''AU''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/QBSH_uitdenbogerd.pdf Alexandra Uitdenbogerd]<br />
+'''AU''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_uitdenbogerd.pdf Alexandra Uitdenbogerd]<br />
-'''CS1''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/QBSH_sailer.pdf Christian Sailer-ear]<br />
+'''CS1''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_sailer.pdf Christian Sailer-ear]<br />
-'''CS2''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/QBSH_sailer.pdf Christian Sailer-midi]<br />
+'''CS2''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_sailer.pdf Christian Sailer-midi]<br />
-'''CS3''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/QBSH_sailer.pdf Christian Sailer-warp]<br />
+'''CS3''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_sailer.pdf Christian Sailer-warp]<br />
 '''FH''' = Pascal Ferraro and Pierre Hanna<br />
-'''NM''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/QBSH_mikkila.pdf Kjell Lemstr├╢m, Niko Mikkil├ñ, Veli M├ñkinen and Esko Ukkonen]<br />
+'''NM''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_mikkila.pdf Kjell Lemstr├╢m, Niko Mikkil├ñ, Veli M├ñkinen and Esko Ukkonen]<br />
-'''RJ''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/QBSH_jang.pdf Jyh-Shing Roger Jang and Nien-Jung Lee]<br />
+'''RJ''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_jang.pdf Jyh-Shing Roger Jang and Nien-Jung Lee]<br />
-'''RL''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/QBSH_tararira.pdf Ernesto Lopez and Mart├¡n Rocamora]<br />
+'''RL''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_tararira.pdf Ernesto Lopez and Mart├¡n Rocamora]<br />
-'''RT''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/SMS_QBSH_typke.pdf Rainer Typke, Frans Wiering and Remco C. Veltkamp]<br />
+'''RT''' = [https://www.music-ir.org/mirex/abstracts/2006/SMS_QBSH_typke.pdf Rainer Typke, Frans Wiering and Remco C. Veltkamp]<br />
-'''XW''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/QBSH_wu.pdf Xiao Wu and Ming Li]<br />
+'''XW''' = [https://www.music-ir.org/mirex/abstracts/2006/QBSH_wu.pdf Xiao Wu and Ming Li]<br />
 ===Calculating Summary Measures===

@@ Line 24: / Line 24: @@
 ==Overall Summary Results==
-<csv>qbsh_overall_sum.csv</csv>
+<csv>2006/qbsh_overall_sum.csv</csv>
 ===QBSH Task 1 Runtime Data===
-<csv>qbsh06_task1_runtime.csv</csv>
+<csv>2006/qbsh06_task1_runtime.csv</csv>
 ===QBSH Task 2 Runtime Data===
-<csv>qbsh06_task2_runtime.csv</csv>
+<csv>2006/qbsh06_task2_runtime.csv</csv>
 ==Task I: "Known-Item Searching" Friedman Test with Multiple Comparisons Results (p=0.05)==
 The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups.<br />
 Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
-<csv>qush_friedman.csv</csv>
+<csv>2006/qush_friedman.csv</csv>
-<csv>QBSH_sum.csv</csv>
+<csv>2006/QBSH_sum.csv</csv>
 ==Task I: "Known-Item Searching" Summary Results==
 MMR data summarized by the 48 ground truth song groups.
-<csv>qbsh_task1_sum.csv</csv>
+<csv>2006/qbsh_task1_sum.csv</csv>
 ==Task II: "Queries as Variations" Friedman Test with Multiple Comparisons Results (p=0.05)==
@@ Line 48: / Line 48: @@
 The Friedman test was run in MATLAB against the QBSH Task 2 mean precision data over the 48 ground truth song groups.<br />
 Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
-<csv>qbsh2_friedman.csv</csv>
+<csv>2006/qbsh2_friedman.csv</csv>
-<csv>qbsh2_sum.csv</csv>
+<csv>2006/qbsh2_sum.csv</csv>
 ==Task II: "Queries as Variations" Summary Results==
 Mean precision data summarized over the 48 ground truth song groups.
-<csv>qbsh06_task2_sum.csv</csv>
+<csv>2006/qbsh06_task2_sum.csv</csv>
 ==Raw Scores==
 The raw data are located on the [[2006:Query-by-Singing/Humming Raw Data]] page.

← Older revision		Revision as of 18:05, 13 May 2010
Line 56:		Line 56:

	==Raw Scores==		==Raw Scores==
−	The raw data are located on the [[Query-by-Singing/Humming Raw Data]] page.	+	The raw data are located on the [[2006:Query-by-Singing/Humming Raw Data]] page.

@@ Line 4: / Line 4: @@
 QBSH task consists of two subtasks. The first subtask is known as '''Known-Item Retrieval'''. In this subtask, submitted systems take a sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth is calculated over the top 20 returns. The test database consists of 48 ground-truth MIDIs + 2000 Essen Collection MIDI noise files. The query database consists of 2797 sung queries.
-The second subtask is called '''Variants Retrieval'''. In this subtask, systems take an input from the query database which consists of 2797 sung queries + 48 ground truth files and return a list of 20 songs from the test database which consists of 48 ground truth MIDIs + 2000 Essen MIDI noise files + 2797 sung queries. The precision based on the number of songs within the same ground truth class of the query is calculated over the top 20 returns.
+The second subtask is called '''Queries as Variations'''. In this subtask, systems take an input from the query database which consists of 2797 sung queries + 48 ground truth files and return a list of 20 songs from the test database which consists of 48 ground truth MIDIs + 2000 Essen MIDI noise files + 2797 sung queries. The precision based on the number of songs within the same ground truth class of the query is calculated over the top 20 returns for each of the 2845 queries.
 ===General Legend===