2006:Symbolic Melodic Similarity Results - Revision history

Singh14: /* Team ID */

2010-07-26T17:07:16Z

‎Team ID

IMIRSELBot: Robot: Automated text replacement (-http://www.music-ir.org/evaluation/MIREX/2006_abstracts/ +http://www.music-ir.org/mirex/abstracts/2006/)

2010-05-14T01:33:28Z

Robot: Automated text replacement (-http://www.music-ir.org/evaluation/MIREX/2006_abstracts/ +http://www.music-ir.org/mirex/abstracts/2006/)

IMIRSELBot: Robot: Automated text replacement (-]*)> +2006/)

2010-05-14T00:38:25Z

Robot: Automated text replacement (-<csv([^>]*)> +<csv\1>2006/)

IMIRSELBot: Robot: Automated text replacement (-\[\[([A-Z][^:]+)\]\] +2006:\1)

2010-05-13T18:06:24Z

Robot: Automated text replacement (-\[\[([A-Z][^:]+)\]\] +2006:\1)

Admin: /* Introduction */

2006-10-03T23:45:30Z

‎Introduction

Mertbay at 23:42, 3 October 2006

2006-10-03T23:42:41Z

Admin: /* Introduction */

2006-10-02T14:54:44Z

‎Introduction

Admin: /* Introduction */

2006-10-02T14:53:09Z

‎Introduction

Admin: /* Introduction */

2006-10-02T14:52:13Z

‎Introduction

Admin: /* Introduction */

2006-10-02T14:51:15Z

‎Introduction

@@ Line 18: / Line 18: @@
 '''FH''' = [https://www.music-ir.org/mirex/abstracts/2006/SMS_hanna.pdf Pascal Ferraro and Pierre Hanna]<br />
-'''NM''' = [https://www.music-ir.org/mirex/abstracts/2006/SMS_mikkila.pdf Kjell Lemstr├╢m, Niko Mikkil├ñ, Veli M├ñkinen and Esko Ukkonen]<br />
+'''NM''' = [https://www.music-ir.org/mirex/abstracts/2006/SMS_mikkila.pdf Kjell Lemström, Niko Mikkilä, Veli Mäkinen and Esko Ukkonen]<br />
 '''RT''' = [https://www.music-ir.org/mirex/abstracts/2006/SMS_QBSH_typke.pdf Rainer Typke, Frans Wiering and Remco C. Veltkamp]<br />
 '''KF''' = [https://www.music-ir.org/mirex/abstracts/2006/SMS_frieler.pdf Klaus Frieler]<br />

@@ Line 17: / Line 17: @@
 Prefix '''R''' = RISM collection, '''K''' = Karaoke collection, '''M''' = Polyphonic collection<br />
-'''FH''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/SMS_hanna.pdf Pascal Ferraro and Pierre Hanna]<br />
+'''FH''' = [https://www.music-ir.org/mirex/abstracts/2006/SMS_hanna.pdf Pascal Ferraro and Pierre Hanna]<br />
-'''NM''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/SMS_mikkila.pdf Kjell Lemstr├╢m, Niko Mikkil├ñ, Veli M├ñkinen and Esko Ukkonen]<br />
+'''NM''' = [https://www.music-ir.org/mirex/abstracts/2006/SMS_mikkila.pdf Kjell Lemstr├╢m, Niko Mikkil├ñ, Veli M├ñkinen and Esko Ukkonen]<br />
-'''RT''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/SMS_QBSH_typke.pdf Rainer Typke, Frans Wiering and Remco C. Veltkamp]<br />
+'''RT''' = [https://www.music-ir.org/mirex/abstracts/2006/SMS_QBSH_typke.pdf Rainer Typke, Frans Wiering and Remco C. Veltkamp]<br />
-'''KF''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/SMS_frieler.pdf Klaus Frieler]<br />
+'''KF''' = [https://www.music-ir.org/mirex/abstracts/2006/SMS_frieler.pdf Klaus Frieler]<br />
-'''AU''' = [https://www.music-ir.org/evaluation/MIREX/2006_abstracts/SMS_uitdenbogerd.pdf Alexandra Uitdenbogerd]<br />
+'''AU''' = [https://www.music-ir.org/mirex/abstracts/2006/SMS_uitdenbogerd.pdf Alexandra Uitdenbogerd]<br />
 ====Broad Categories====

@@ Line 51: / Line 51: @@
 ===Task I: RISM Overall Summary===
-<csv>sms06_rism_sum.csv</csv>
+<csv>2006/sms06_rism_sum.csv</csv>
 ===Task I: RISM Runtime Data===
-<csv>sms06_rism_runtime.csv</csv>
+<csv>2006/sms06_rism_runtime.csv</csv>
 ===Task IIa: Karaoke Overall Summary===
-<csv>sms06_karaoke_sum.csv</csv>
+<csv>2006/sms06_karaoke_sum.csv</csv>
 ===Task IIa: Karaoke Runtime Data===
-<csv>sms06_karaoke_runtime.csv</csv>
+<csv>2006/sms06_karaoke_runtime.csv</csv>
 ===Task IIb: Mixed Polyphonic Overall Summary===
-<csv>sms06_mixed_sum.csv</csv>
+<csv>2006/sms06_mixed_sum.csv</csv>
 ===Task IIb: Mixed Polyphonic Runtime Data===
-<csv>sms06_mixed_runtime.csv</csv>
+<csv>2006/sms06_mixed_runtime.csv</csv>
 ==Task I: RISM Collection Summary Results==
 There is an error with this data set...please stand by.
-<csv>sms06_rism_results3.csv</csv>
+<csv>2006/sms06_rism_results3.csv</csv>
 ==Task IIa: Karaoke Collection Summary Results==
-<csv>sms06_kar_results3.csv</csv>
+<csv>2006/sms06_kar_results3.csv</csv>
 ==Task IIb: Mixed Polyphonic Collection Summary Results==
-<csv>sms06_mix_results3.csv</csv>
+<csv>2006/sms06_mix_results3.csv</csv>
 ==Raw Scores==
 The raw data derived from the Evalutron 6000 human evaluations are located on the [[2006:Symbolic Melodic Similarity Raw Data]] page.

@@ Line 1: / Line 1: @@
 [[Category: Results]]
 ==Introduction==
-These are the results for the 2006 running of the Symbolic Melodic Similarity task set. For background information about this task set please refer to the [[Symbolic Melodic Similarity]] page.
+These are the results for the 2006 running of the Symbolic Melodic Similarity task set. For background information about this task set please refer to the [[2006:Symbolic Melodic Similarity]] page.
 Each system was given a query and returned the 10 most melodically similar songs from a given collection where the collections were RISM (monophonic; 10,000), Karoke (polyphonic; 1,000), Mixed (polyphonic; 15,741). Then, for each query, the returned results from all participants were grouped and were evaluated by human graders, each query being evaluated by 3 different graders with two scores (using the Evalutron 6000 system). Graders were asked to provide 1 categorical score with 3 categories: NS,SS,VS as explained below, and one fine score (in the range from 0 to 10).
@@ Line 46: / Line 46: @@
 ==Overall Summary Results==
 ===Visualizations===
-Rainer Typke has created a series of [[Symbolic Melodic Similarity Graphs]] that help us visualize the results. <br />
+Rainer Typke has created a series of [[2006:Symbolic Melodic Similarity Graphs]] that help us visualize the results. <br />
 Rainer Typke has also created a set of detailed representations of the results that is definitely with exploring at [[http://rainer.typke.org/mirex06.0.html http://rainer.typke.org/mirex06.0.html]].
@@ Line 86: / Line 86: @@
 ==Raw Scores==
-The raw data derived from the Evalutron 6000 human evaluations are located on the [[Symbolic Melodic Similarity Raw Data]] page.
+The raw data derived from the Evalutron 6000 human evaluations are located on the [[2006:Symbolic Melodic Similarity Raw Data]] page.

@@ Line 2: / Line 2: @@
 ==Introduction==
 These are the results for the 2006 running of the Symbolic Melodic Similarity task set. For background information about this task set please refer to the [[Symbolic Melodic Similarity]] page.
-Each system was given a query and returned the 10 most melodically similar songs from a given collection where the collections were rism (monophonic), karoke(polyphonic), mix (polyphonic). Then, for each query, the returned results from all participants were grouped and were evaluated by human graders, each query being evaluated by 3 different graders with two scores (using the Evalutron 6000 system). Graders were asked to provide 1 categorical score with 3 categories: NS,SS,VS as explained below, and one fine score (in the range from 0 to 10).
 ====Evalutron 6000 Summary Data====
-'''Number of evaluators''' = 2x<br />
+'''Number of evaluators''' = 20<br />
 '''Number of evaluations per query/candidate pair''' = 3<br />
 '''Number of queries per grader''' = 15<br />

@@ Line 8: / Line 8: @@
 '''Number of queries per grader''' = 15<br />
 '''Ave. size of the candidate lists''' = 15<br />
-'''Ave. number of query/candidate pairs evaluated per grader: 225
+'''Ave. number of query/candidate pairs evaluated per grader: 225<br />
 '''Number of queries (across all subtasks''' = 17<br />

@@ Line 7: / Line 7: @@
 '''Number of queries per grader''' = 15<br />
 '''Size of the candidate lists''' = 15<br />
-'''Number of queries''' = 17<br />
+'''Ave. number of query/candidate pairs evaluated per grader: 225
 ===General Legend===