2009:Audio Chord Detection - Revision history

Kahyun Choi: /* Discussions */

2011-12-15T06:44:36Z

‎Discussions

Show changes

Kahyun Choi: /* Evaluation */

2011-12-15T06:41:30Z

‎Evaluation

Kahyun Choi: /* Evaluation */

2011-12-15T06:40:55Z

‎Evaluation

Chrish: /* Evaluation */

2009-09-09T17:05:47Z

‎Evaluation

Chrish: /* Discussions */

2009-09-09T17:04:00Z

‎Discussions

Chrish: /* Segmentation Score */

2009-09-09T16:47:25Z

‎Segmentation Score

Chrish: /* Evaluation */

2009-09-09T16:45:35Z

‎Evaluation

MertBay: /* Evaluation */

2009-09-08T23:18:12Z

‎Evaluation

MertBay: /* Evaluation */

2009-09-08T21:49:59Z

‎Evaluation

MertBay: /* Evaluation */

2009-09-08T21:49:27Z

‎Evaluation

@@ Line 187: / Line 187: @@
 with elements separated by white spaces, times given in seconds, chord labels corresponding to the syntax described in [6] and one chord segment per line.
-Please note that two things have changed in the syntax since it was originally described in [6]. The first change is that the root is no longer implied as a voiced element of a chord so a C major chord (notes C, E and G) should be written C:(1,3,5) instead of just C:(3,5) if using the interval list representation. As before, the labels C and C:maj are equivalent to C:(1,3,5). The second change is that the shorthand label "sus2" (intervals 1,2,5) has been added to the available shorthand list.--[[User:Chrish|Chrish]] 17:05, 9 September 2009 (UTC)
+Please note that two things have changed in the syntax since it was originally described in [6]. The first change is that the root is no longer implied as a voiced element of a chord so a C major chord (notes C, E and G) should be written C:(1,3,5) instead of just C:(3,5) if using the interval list representation. As before, the labels C and C:maj are equivalent to C:(1,3,5). The second change is that the shorthand label "sus2" (intervals 1,2,5) has been added to the available shorthand list.--Chrish 17:05, 9 September 2009 (UTC)
@@ Line 196: / Line 196: @@
 segmentation score = 1 - max(m,f)
-m and f are not independent of each other so combining them this way ensures that a good score in one does not hide a bad score in the other. The combined segmentation score can take values between 0 and 1 with 0 being the worst and 1 being the best result.--[[User:Chrish|Chrish]] 17:05, 9 September 2009 (UTC)
+m and f are not independent of each other so combining them this way ensures that a good score in one does not hide a bad score in the other. The combined segmentation score can take values between 0 and 1 with 0 being the worst and 1 being the best result.-- Chrish 17:05, 9 September 2009 (UTC)
 === Frame-based recall ===

@@ Line 277: / Line 277: @@
 ) go back to 4 until final chord frame
---[[User:Chrish|Chrish]] 17:05, 9 September 2009 (UTC)
+--Chrish 17:05, 9 September 2009 (UTC)
 == Discussions ==

@@ Line 187: / Line 187: @@
 with elements separated by white spaces, times given in seconds, chord labels corresponding to the syntax described in [6] and one chord segment per line.
-Please note that two things have changed in the syntax since it was originally described in [6]. The first change is that the root is no longer implied as a voiced element of a chord so a C major chord (notes C, E and G) should be written C:(1,3,5) instead of just C:(3,5) if using the interval list representation. As before, the labels C and C:maj are equivalent to C:(1,3,5). The second change is that the shorthand label "sus2" (intervals 1,2,5) has been added to the available shorthand list.
+Please note that two things have changed in the syntax since it was originally described in [6]. The first change is that the root is no longer implied as a voiced element of a chord so a C major chord (notes C, E and G) should be written C:(1,3,5) instead of just C:(3,5) if using the interval list representation. As before, the labels C and C:maj are equivalent to C:(1,3,5). The second change is that the shorthand label "sus2" (intervals 1,2,5) has been added to the available shorthand list.--[[User:Chrish|Chrish]] 17:05, 9 September 2009 (UTC)
@@ Line 196: / Line 196: @@
 segmentation score = 1 - max(m,f)
-m and f are not independent of each other so combining them this way ensures that a good score in one does not hide a bad score in the other. The combined segmentation score can take values between 0 and 1 with 0 being the worst and 1 being the best result.
+m and f are not independent of each other so combining them this way ensures that a good score in one does not hide a bad score in the other. The combined segmentation score can take values between 0 and 1 with 0 being the worst and 1 being the best result.--[[User:Chrish|Chrish]] 17:05, 9 September 2009 (UTC)
 === Frame-based recall ===
@@ Line 277: / Line 277: @@
 ) go back to 4 until final chord frame
 == Discussions ==

@@ Line 307: / Line 307: @@
 * Should chord data be expressed in absolute (aka "F major-minor 7") or relative (aka "C: IV major-minor 7") terms?
 * Should different inversions of chords be considered in the evaluation process?
 * What temporal resolution should be used for ground truth and results?
 * How should enharmonic and other confusions of chords be handled?
 * How will Ground Truth be determined?
 *# One thing we should consider doing is using the combined results to find possible errors or inconsistencies in the data.  I have found that there are some that exists in Harte's labels (this was, after all, extremely difficult).  If there are sections that are systematically error-prone, then maybe it is best to reevaluate these labels for future runs  Also, we need to start finding other databases.  Since we are only using Beatles data each year, we have no measure of generalization.  Maybe we should make a requirement that participation includes a requirement to label 2-3 songs?  These could be distributed to a few people and for verification as well.  Just a thought.  -- [[User:Jeremy|Jeremy]] 14:45, 25 August 2009 (UTC)
 * What degree of chordal/tonal complexity will the music contain?
 * Will we include any atonal or polytonal music in the Ground Truth dataset?
 * What is the maximal acceptable onset deviation between ground truth and result?
 * What file format should be used for ground truth and output?
 === Comment`s by MB 08.12===

@@ Line 192: / Line 192: @@
 === Segmentation Score ===
-The segmentation score will be calculated using directional hamming distance as described in [8]. An over-segmentation value (m) and an under-segmentation value (f) will be calculated and the final segmentation score will be taken as the worst case from these two i.e:
+The segmentation score will be calculated using directional hamming distance as described in [8]. An over-segmentation value (m) and an under-segmentation value (f) will be calculated and the final segmentation score will be calculated using the worst case from these two i.e:
 segmentation score = 1 - max(m,f)

@@ Line 181: / Line 181: @@
 == Evaluation ==
-frame-based recall measure based on .01s frames.
+start_time end_time chord_label
-Chord segmentation score between the ground-truth and estimated annotations based on directional hamming distance.
+with elements separated by white spaces, times given in seconds, chord labels corresponding to the syntax described in [6] and one chord segment per line.
-Overlap score based on the amount overlap with the ground-truth.
+Please note that two things have changed in the syntax since it was originally described in [6]. The first change is that the root is no longer implied as a voiced element of a chord so a C major chord (notes C, E and G) should be written C:(1,3,5) instead of just C:(3,5) if using the interval list representation. As before, the labels C and C:maj are equivalent to C:(1,3,5). The second change is that the shorthand label "sus2" (intervals 1,2,5) has been added to the available shorthand list.
-Algorithms will be evaluated in different levels based on the dictionary they used.  The first level will be maj/min no chord.  The second level will be maj, min, aug, dim sus2 and sus4. The third level will be all quartets. Chords in the ground-truth that are not in the dictionary will be ignored.
 == Discussions ==

← Older revision		Revision as of 23:18, 8 September 2009
Line 182:		Line 182:


−	frame-based recall measure based on .1s frames.	+	frame-based recall measure based on .01s frames.

	Chord segmentation score between the ground-truth and estimated annotations based on directional hamming distance.		Chord segmentation score between the ground-truth and estimated annotations based on directional hamming distance.

@@ Line 186: / Line 186: @@
 Chord segmentation score between the ground-truth and estimated annotations based on directional hamming distance.
-Overlap score based on the amounth overlap with the ground-truth.
+Overlap score based on the amount overlap with the ground-truth.
 Algorithms will be evaluated in different levels based on what the dictionary.  The first level will be maj/min no chord. Chords that are not in the dictionary will be ignored. The second level will be maj, min, aug, dim sus2 and sus4. The third level will be all quartets.