2006:Symbolic Melodic Similarity - Revision history

IMIRSELBot: Robot: Automated text replacement (-\[\[([A-Z][^:]+)\]\] +2006:\1)

2010-05-13T18:06:04Z

Robot: Automated text replacement (-\[\[([A-Z][^:]+)\]\] +2006:\1)

Xiao: /* Results */

2006-10-06T22:53:57Z

‎Results

Xiao: /* Overview */

2006-10-06T22:53:33Z

‎Overview

Alu: /* Alexandra Uitdenbogerd */

2006-08-13T11:13:25Z

‎Alexandra Uitdenbogerd

Alu: /* Alexandra Uitdenbogerd */

2006-08-13T11:13:00Z

‎Alexandra Uitdenbogerd

Alu: /* Alexandra Uitdenbogerd */

2006-08-13T11:12:36Z

‎Alexandra Uitdenbogerd

Alu: /* Alexandra Uitdenbogerd */

2006-08-13T11:11:27Z

‎Alexandra Uitdenbogerd

Alu: /* Rainer Typke */

2006-08-13T11:10:51Z

‎Rainer Typke

Rtypke: /* Rainer Typke */

2006-08-08T10:49:20Z

‎Rainer Typke

Rtypke at 10:44, 8 August 2006

2006-08-08T10:44:09Z

@@ Line 1: / Line 1: @@
 ==Results==
-Results are on [[Symbolic Melodic Similarity Results]] page.
+Results are on [[2006:Symbolic Melodic Similarity Results]] page.
 == Overview ==

@@ Line 1: / Line 1: @@
 ==Results==
-Results are on [[Score Following Results]] page.
+Results are on [[Symbolic Melodic Similarity Results]] page.
 == Overview ==

← Older revision		Revision as of 22:53, 6 October 2006
Line 1:		Line 1:
		+	==Results==
		+
		+	Results are on [[Score Following Results]] page.
		+
	== Overview ==		== Overview ==

@@ Line 95: / Line 95: @@
 <p>
 I agree that it may not work with hummed queries.  I haven't tried it.
-My n-gram technique was tested on melodies with errors, so may be reasonably robust, but the errors were not transcribed singing type errors.
+My n-gram technique was tested on melodies with errors, so may be reasonably robust, but the errors were not transcribed singing type errors.  </p>
 <p>
-However, on reading this year's requirements, it wouldn't be possible to just run last year's code on this year's set.
+However, on reading this year's requirements, it wouldn't be possible to just run last year's code on this year's set.  </p>
 <p>
-Some may argue that their algorithms are designed for the new task and not the original one (or vice versa), so may object to running them on both collections.  I would be happy for mine to run on both, out of curiosity.
+Some may argue that their algorithms are designed for the new task and not the original one (or vice versa), so may object to running them on both collections.  I would be happy for mine to run on both, out of curiosity.</p>

@@ Line 95: / Line 95: @@
 <p>
 I agree that it may not work with hummed queries.  I haven't tried it.
-My n-gram technique was tested on melodies with errors, so may be reasonably robust, but the errors were not transcribed singing type errors.  <p>
+My n-gram technique was tested on melodies with errors, so may be reasonably robust, but the errors were not transcribed singing type errors.
-However, on reading this year's requirements, it wouldn't be possible to just run last year's code on this year's set.  <p>
+<p>
 Some may argue that their algorithms are designed for the new task and not the original one (or vice versa), so may object to running them on both collections.  I would be happy for mine to run on both, out of curiosity.

← Older revision		Revision as of 11:10, 13 August 2006
Line 90:		Line 90:

	I am copying these last two bits of discussion to the mailing list, to make sure that people who don't monitor this page can participate in the discussion.		I am copying these last two bits of discussion to the mailing list, to make sure that people who don't monitor this page can participate in the discussion.
		+
		+	=== Alexandra Uitdenbogerd ===
		+	I agree that it may not work with hummed queries. I haven't tried it.
		+	My n-gram technique was tested on melodies with errors, so may be reasonably robust, but the errors were not transcribed singing type errors.
		+	However, on reading this year's requirements, it wouldn't be possible to just run last year's code on this year's set.
		+	Some may argue that their algorithms are designed for the new task and not the original one (or vice versa), so may object to running them on both collections. I would be happy for mine to run on both, out of curiosity.

@@ Line 83: / Line 83: @@
 ===Rainer Typke===
-What do you mean with "using as a baseline"? A replacement for the ground truth? I don't think that would work all that well with n-grams because this year, we are working with hummed queries as well as exact ones (half the queries will have perfect rhythm and pitch, like last year, but the other half will be hummed; note that by having "perfect rhythm and pitch", I don't mean that they are guaranteed to be error-free - they are still entered by a human user). For the hummed and therefore distorted queries, n-grams would fail unless you do some quantization first, which could introduce errors.
+What do you mean with "using as a baseline"? A replacement for the ground truth? I don't think that would work all that well with n-grams because this year, we are working with hummed queries as well as exact ones (half the queries will have perfect rhythm and pitch, like last year, but the other half will be hummed; note that by having "perfect rhythm and pitch", I don't mean that they are guaranteed to be error-free - for the non-RISM tasks, they are still entered by a human user). For the hummed and therefore distorted queries, n-grams would fail unless you do some quantization first, which could introduce errors.
 So, since the nature of the task changed a bit, I am not sure if knowing how we perform compared to one particular algorithm will tell us much. That reference algorithm might just be better at last year's task than this year's, which might lead us to believe that we got better even if we didn't.

← Older revision		Revision as of 10:44, 8 August 2006
Line 80:		Line 80:
	give us an idea of whether we're progressing or just submitting different		give us an idea of whether we're progressing or just submitting different
	algorithms each year that do about the same.		algorithms each year that do about the same.
		+
		+
		+	===Rainer Typke===
		+	What do you mean with "using as a baseline"? A replacement for the ground truth? I don't think that would work all that well with n-grams because this year, we are working with hummed queries as well as exact ones (half the queries will have perfect rhythm and pitch, like last year, but the other half will be hummed; note that by having "perfect rhythm and pitch", I don't mean that they are guaranteed to be error-free - they are still entered by a human user). For the hummed and therefore distorted queries, n-grams would fail unless you do some quantization first, which could introduce errors.
		+
		+	So, since the nature of the task changed a bit, I am not sure if knowing how we perform compared to one particular algorithm will tell us much. That reference algorithm might just be better at last year's task than this year's, which might lead us to believe that we got better even if we didn't.
		+
		+	To really compare the results with last year, IMO, we would have to let all algorithms perform the same task as last year in addition to this year's harder tasks. This might actually be sensible since to my knowledge, last year's evaluation collection and queries were never made public, so the new algorithms can't be easily over-trained for last year's task.
		+
		+	I am copying these last two bits of discussion to the mailing list, to make sure that people who don't monitor this page can participate in the discussion.