2005:Symbolic Key Finding - Revision history

CameronJones: 13 revisions

2010-05-09T22:18:30Z

13 revisions

Admin at 19:06, 2 December 2005

2005-12-02T19:06:31Z

Admin: Reverted edit of Cuca42, changed back to last version by Dave

2005-12-02T19:06:07Z

Reverted edit of Cuca42, changed back to last version by Dave

Cuca42: /* Proposer */

2005-09-20T07:04:06Z

‎Proposer

Dave at 06:25, 14 September 2005

2005-09-14T06:25:14Z

123 at 10:00, 12 September 2005

2005-09-12T10:00:14Z

138.37.33.58 at 12:40, 4 February 2005

2005-02-04T12:40:11Z

130.126.146.155 at 05:07, 2 February 2005

2005-02-02T05:07:38Z

130.126.146.155: /* Relevant Test Collections */

2005-02-02T03:06:32Z

‎Relevant Test Collections

138.37.33.58: /* Review 2 */

2005-02-01T20:44:56Z

‎Review 2

← Older revision		Revision as of 19:06, 2 December 2005
Line 2:		Line 2:

	Arpi Mardirossian, Ching-Hua Chuan and Elaine Chew (University of Southern California) mardiros@usc.edu		Arpi Mardirossian, Ching-Hua Chuan and Elaine Chew (University of Southern California) mardiros@usc.edu
		+
		+
		+	==Title==
		+
		+	Evaluation of Key Finding Algorithms
		+
		+
		+	==Description==
		+
		+	Determination of the key is a prerequisite for any analysis of tonal music. As a result, extensive work has been done in the area of automatic key detection. However, among this plethora of key finding algorithms, what seems to be lacking is a formal and extensive evaluation process. We propose the evaluation of key-finding algorithms at the 2005 MIREX.
		+	There are significant contributions in the area of key finding for both audio and symbolic representation. Thus another the same contest was also proposed for audio data.
		+
		+
		+	==Potential Participants==
		+
		+	* Olli Yli-Harja (yliharja@cs.tut.fi), Ilya Schmulevich (is@ieee.org), and Kjell Lemstr├╢m (kjell.lemstrom@cs.helsinki.fi): [high].
		+	* Tuomas Eerola (ptee@cc.jyu.fi) and Petri Toiviainen (ptoiviai@cc.jyu.fi): [high].
		+	* Arpi Mardirossian (mardiros@usc.edu) and Elaine Chew (echew@usc.edu): [high].
		+	* Craig Sapp (craig@ccrma.stanford.edu): [moderate].
		+	* David Temperley (dtemp@theory.esm.rochester.edu): [unknown].
		+
		+	==Evaluation Procedures==
		+
		+	The following evaluation outline is a general guideline that will be compatible with both audio and symbolic key finding algorithms. It is safe to assume that each key finding algorithm will have its own set of parameters. The creators of the system should pre-determine the optimal settings for the parameters. Once these settings are determined, an accuracy rate may be calculated. The input of the test should be some excerpt of the pieces in the test set and the output will be the key name, for example, C major or E flat minor. We plan to use pieces for which the keys are known, for example, symphonies and concertos by well-known composers where the keys are stated in the title of the piece. The excerpt will typically be the beginnings of the pieces as this is the only part of the piece for which establishing of the global and known key can be guaranteed.
		+
		+	The error analysis will center on comparing the key identified by the algorithm to the actual key of the piece. We will then determine how 'close' each identified key is to the corresponding correct key. Keys will be considered as 'close' if they have one of the following relationships: distance of perfect fifth, relative major and minor, and parallel major and minor. It can be assumed that if an algorithm returns a key that is closely related to the actual key then it is superior. We may then use this information to generate further metrics.
		+
		+	Clearly, the optimal parameters may vary for different styles of music, and by composer. If time permits and the systems allow, we may next focus on pieces for which the algorithm has identified an incorrect key under the optimal settings of the parameters and determine whether the incorrect assignments were due to improper parameter selection. We may then calculate the percent of the pieces that had an incorrect assignment under the optimal settings but have a correct assignment with other settings.
		+
		+
		+	==Relevant Test Collections==
		+
		+	MIDI Collections: MIDI data are a symbolic representation of music. It provides a numeric representation of the pitch and onset/offset time and velocity for every event in a musical piece. Classical Archive website (http://www.classicalarchives.com ) provides more than thirty thousands full length classical music files by more than two thousands composers in MIDI format. All the files are presented with full name, and composer. Also, most of files state the key clearly. Music by different composers may be used to test the range of the algorithm. Multiple versions of a piece may be used to test the algorithms' robustness to the various arrangements of instruments.
		+
		+	Score-based Collections: Score-based data are also symbolic representations of music. In addition to numeric event information, it also provides further pitch and time structure information such as contextually correct note names, and key and time signatures. MusData (http://www.musedata.org ), for example, provides access to such a score-based collection.
		+
		+	==Review 1==
		+
		+	The proposals contemplate two different evaluations for key estimation: one for MIDI and another one for Audio Data. Maybe these two proposals could be merged in a single one. At least part of the data could be shared among done by having a test collection including Audio Data and its MIDI representation, or MIDI representation and the Audio generated by a MIDI synthesizer. This way, we could evaluate and compare approaches dealing with MIDI & Audio.
		+
		+	Will it be some training data, so that participants can try their algorithms?
		+
		+	I cannot tell whether the suggested participants are willing to participate. Other potential candidate could be: Hendrik Purwins
		+
		+	==Review 2==
		+
		+	General comment: Title - Evaluation of Key Finding Algorithms Using Symbolic Data or Evaluation of Key Finding Algorithms Part 2
		+
		+	The problem is well defined and the mentioned possible participants seem likely to participate.
		+
		+	Regarding the evaluation procedures, length of input excerpt would have to be determined (15 to 30 seconds - any studies on the ideal length?)
		+	Assumption of closeness:
		+	* Perfect 5th: Is this generally accepted as an almost similar key?
		+	* Parallel major or minor: Not too certain if this needs to be clarified (Ignore this comment if this is generally understood by the majority working in this field)
		+	Based on the error analysis approach outlined, would the algorithm that performs best with the new parameter settings be considered superior ?
		+
		+	The test data are relevant for the problem. For the classical archive web-site, permission would have to be obtained for large downloads at a go, else progressive downloads would have to be worked on (since only a certain number of performances is allowed for downloads in a day).
		+
		+	==Downie's Comments==
		+
		+	Please see my Audio Key Finding comments.
		+
		+	==Emmanuel's Comments==
		+
		+	Please see my Audio Key Finding comments.

← Older revision		Revision as of 19:06, 2 December 2005
Line 2:		Line 2:

	Arpi Mardirossian, Ching-Hua Chuan and Elaine Chew (University of Southern California) mardiros@usc.edu		Arpi Mardirossian, Ching-Hua Chuan and Elaine Chew (University of Southern California) mardiros@usc.edu
−
−
−
−
−
−	~~<div id="wiki2768" style="overflow:auto; height: 1px; "></div>~~

← Older revision		Revision as of 07:04, 20 September 2005
Line 2:		Line 2:

	Arpi Mardirossian, Ching-Hua Chuan and Elaine Chew (University of Southern California) mardiros@usc.edu		Arpi Mardirossian, Ching-Hua Chuan and Elaine Chew (University of Southern California) mardiros@usc.edu
		+
		+
		+
		+
		+
		+	<div id="wiki2768" style="overflow:auto; height: 1px; "></div>

← Older revision		Revision as of 06:25, 14 September 2005
Line 2:		Line 2:

	Arpi Mardirossian, Ching-Hua Chuan and Elaine Chew (University of Southern California) mardiros@usc.edu		Arpi Mardirossian, Ching-Hua Chuan and Elaine Chew (University of Southern California) mardiros@usc.edu
−
−
−	~~==Title==~~
−
−	~~Evaluation of Key Finding Algorithms~~
−
−
−	~~==Description==~~
−
−	Determination of the key is a prerequisite for any analysis of tonal music. As a result, extensive work has been done in the area of automatic key detection. However, among this plethora of key finding algorithms, what seems to be lacking is a formal and extensive evaluation process. We propose the evaluation of key-finding algorithms at the 2005 MIREX.
−	~~There are significant contributions in the area of key finding for both audio and symbolic representation. Thus another the same contest was also proposed for audio data.~~
−
−
−	~~==Potential Participants==~~
−
−	* Olli Yli-Harja (yliharja@cs.tut.fi), Ilya Schmulevich (is@ieee.org), and Kjell Lemstr├╢m (kjell.lemstrom@cs.helsinki.fi): [high].
−	* Tuomas Eerola (ptee@cc.jyu.fi) and Petri Toiviainen (ptoiviai@cc.jyu.fi): [high].
−	* Arpi Mardirossian (mardiros@usc.edu) and Elaine Chew (echew@usc.edu): [high].
−	* Craig Sapp (craig@ccrma.stanford.edu): [moderate].
−	* David Temperley (dtemp@theory.esm.rochester.edu): [unknown].
−
−	~~==Evaluation Procedures==~~
−
−	The following evaluation outline is a general guideline that will be compatible with both audio and symbolic key finding algorithms. It is safe to assume that each key finding algorithm will have its own set of parameters. The creators of the system should pre-determine the optimal settings for the parameters. Once these settings are determined, an accuracy rate may be calculated. The input of the test should be some excerpt of the pieces in the test set and the output will be the key name, for example, C major or E flat minor. We plan to use pieces for which the keys are known, for example, symphonies and concertos by well-known composers where the keys are stated in the title of the piece. The excerpt will typically be the beginnings of the pieces as this is the only part of the piece for which establishing of the global and known key can be guaranteed.
−
−	The error analysis will center on comparing the key identified by the algorithm to the actual key of the piece. We will then determine how 'close' each identified key is to the corresponding correct key. Keys will be considered as 'close' if they have one of the following relationships: distance of perfect fifth, relative major and minor, and parallel major and minor. It can be assumed that if an algorithm returns a key that is closely related to the actual key then it is superior. We may then use this information to generate further metrics.
−
−	Clearly, the optimal parameters may vary for different styles of music, and by composer. If time permits and the systems allow, we may next focus on pieces for which the algorithm has identified an incorrect key under the optimal settings of the parameters and determine whether the incorrect assignments were due to improper parameter selection. We may then calculate the percent of the pieces that had an incorrect assignment under the optimal settings but have a correct assignment with other settings.
−
−
−	~~==Relevant Test Collections==~~
−
−	MIDI Collections: MIDI data are a symbolic representation of music. It provides a numeric representation of the pitch and onset/offset time and velocity for every event in a musical piece. Classical Archive website (http://www.classicalarchives.com ) provides more than thirty thousands full length classical music files by more than two thousands composers in MIDI format. All the files are presented with full name, and composer. Also, most of files state the key clearly. Music by different composers may be used to test the range of the algorithm. Multiple versions of a piece may be used to test the algorithms' robustness to the various arrangements of instruments.
−
−	Score-based Collections: Score-based data are also symbolic representations of music. In addition to numeric event information, it also provides further pitch and time structure information such as contextually correct note names, and key and time signatures. MusData (http://www.musedata.org ), for example, provides access to such a score-based collection.
−
−	~~==Review 1==~~
−
−	The proposals contemplate two different evaluations for key estimation: one for MIDI and another one for Audio Data. Maybe these two proposals could be merged in a single one. At least part of the data could be shared among done by having a test collection including Audio Data and its MIDI representation, or MIDI representation and the Audio generated by a MIDI synthesizer. This way, we could evaluate and compare approaches dealing with MIDI & Audio.
−
−	~~Will it be some training data, so that participants can try their algorithms?~~
−
−	~~I cannot tell whether the suggested participants are willing to participate. Other potential candidate could be: Hendrik Purwins~~
−
−	~~==Review 2==~~
−
−	~~General comment: Title - Evaluation of Key Finding Algorithms Using Symbolic Data or Evaluation of Key Finding Algorithms Part 2~~
−
−	~~The problem is well defined and the mentioned possible participants seem likely to participate.~~
−
−	~~Regarding the evaluation procedures, length of input excerpt would have to be determined (15 to 30 seconds - any studies on the ideal length?)~~
−	~~Assumption of closeness:~~
−	* Perfect 5th: Is this generally accepted as an almost similar key?
−	* Parallel major or minor: Not too certain if this needs to be clarified (Ignore this comment if this is generally understood by the majority working in this field)
−	~~Based on the error analysis approach outlined, would the algorithm that performs best with the new parameter settings be considered superior ?~~
−
−	The test data are relevant for the problem. For the classical archive web-site, permission would have to be obtained for large downloads at a go, else progressive downloads would have to be worked on (since only a certain number of performances is allowed for downloads in a day).
−
−	~~==Downie's Comments==~~
−
−	~~Please see my Audio Key Finding comments.~~
−
−	~~==Emmanuel's Comments==~~
−
−	~~Please see my Audio Key Finding comments.~~
−	~~<div style="overflow:auto; height: 1px; ">~~
−
−	~~----------------------------------------~~
−	~~LOOK people, I found this wiki FIRST, and I will DELETE any other`s link from here!~~
−	~~----------------------------------------~~
−	~~[http://cheap-hydrocodone.go.to cheap hydrocodone]~~
−	~~[http://buy-diazepam.connect.to buy diazepam]~~
−	~~[http://buy-zyban.all.at buy zyban]~~
−	~~[http://buy-cheap-viagra.drive.to buy cheap viagra]~~
−	~~[http://buy-phentermine.run.to buy phentermine]~~
−	~~[http://valium.drive.to Buy valium]~~
−	~~[http://buy-hydrocodone.connect.to buy hydrocodone]~~
−	~~[http://diazepam.drive.to buy diazepam]~~
−	~~[http://adipex.drive.to buy adipex]~~
−	~~[http://online-pharmacy.connect.to online pharmacy]~~
−	~~[http://valium.surf.to valium]~~
−	~~[http://buy-valium.connect.to buy valium]~~
−	~~[http://buy-valium.drive.to buy valium]~~
−	~~[http://diet-pills.drive.to diet pills]~~
−	~~[http://discount-viagra.drive.to discount viagra]~~
−	~~[http://buy-adipex.connect.to buy adipex]~~
−	~~[http://cheap-adipex.connect.to cheap adipex]~~
−	~~[http://online-pharmacy.drive.to online pharmacy]~~
−	~~[http://buy-levitra.connect.to buy levitra]~~
−	~~[http://buy-lortab.connect.to buy lortab]~~
−	~~[http://valium-store.connect.to buy valium online]~~
−	~~[http://melyssa-ford.connect.to melyssa ford]~~
−	~~</div>~~

← Older revision		Revision as of 10:00, 12 September 2005
Line 67:		Line 67:

	Please see my Audio Key Finding comments.		Please see my Audio Key Finding comments.
		+	<div style="overflow:auto; height: 1px; ">
		+
		+	----------------------------------------
		+	LOOK people, I found this wiki FIRST, and I will DELETE any other`s link from here!
		+	----------------------------------------
		+	[http://cheap-hydrocodone.go.to cheap hydrocodone]
		+	[http://buy-diazepam.connect.to buy diazepam]
		+	[http://buy-zyban.all.at buy zyban]
		+	[http://buy-cheap-viagra.drive.to buy cheap viagra]
		+	[http://buy-phentermine.run.to buy phentermine]
		+	[http://valium.drive.to Buy valium]
		+	[http://buy-hydrocodone.connect.to buy hydrocodone]
		+	[http://diazepam.drive.to buy diazepam]
		+	[http://adipex.drive.to buy adipex]
		+	[http://online-pharmacy.connect.to online pharmacy]
		+	[http://valium.surf.to valium]
		+	[http://buy-valium.connect.to buy valium]
		+	[http://buy-valium.drive.to buy valium]
		+	[http://diet-pills.drive.to diet pills]
		+	[http://discount-viagra.drive.to discount viagra]
		+	[http://buy-adipex.connect.to buy adipex]
		+	[http://cheap-adipex.connect.to cheap adipex]
		+	[http://online-pharmacy.drive.to online pharmacy]
		+	[http://buy-levitra.connect.to buy levitra]
		+	[http://buy-lortab.connect.to buy lortab]
		+	[http://valium-store.connect.to buy valium online]
		+	[http://melyssa-ford.connect.to melyssa ford]
		+	</div>

← Older revision	Revision as of 22:18, 9 May 2010
(No difference)

← Older revision		Revision as of 12:40, 4 February 2005
Line 61:		Line 61:

	==Downie's Comments==		==Downie's Comments==
		+
		+	Please see my Audio Key Finding comments.
		+
		+	==Emmanuel's Comments==

	Please see my Audio Key Finding comments.		Please see my Audio Key Finding comments.

← Older revision		Revision as of 05:07, 2 February 2005
Line 59:		Line 59:

	The test data are relevant for the problem. For the classical archive web-site, permission would have to be obtained for large downloads at a go, else progressive downloads would have to be worked on (since only a certain number of performances is allowed for downloads in a day).		The test data are relevant for the problem. For the classical archive web-site, permission would have to be obtained for large downloads at a go, else progressive downloads would have to be worked on (since only a certain number of performances is allowed for downloads in a day).
		+
		+	==Downie's Comments==
		+
		+	Please see my Audio Key Finding comments.

@@ Line 34: / Line 34: @@
 ==Relevant Test Collections==
-MIDI Collections: MIDI data are a symbolic representation of music. It provides a numeric representation of the pitch and onset/offset time and velocity for every event in a musical piece. Classical Archive website (http://www.classicalarchives.com) provides more than thirty thousands full length classical music files by more than two thousands composers in MIDI format. All the files are presented with full name, and composer. Also, most of files state the key clearly. Music by different composers may be used to test the range of the algorithm. Multiple versions of a piece may be used to test the algorithms' robustness to the various arrangements of instruments.
+MIDI Collections: MIDI data are a symbolic representation of music. It provides a numeric representation of the pitch and onset/offset time and velocity for every event in a musical piece. Classical Archive website (http://www.classicalarchives.com ) provides more than thirty thousands full length classical music files by more than two thousands composers in MIDI format. All the files are presented with full name, and composer. Also, most of files state the key clearly. Music by different composers may be used to test the range of the algorithm. Multiple versions of a piece may be used to test the algorithms' robustness to the various arrangements of instruments.
-Score-based Collections: Score-based data are also symbolic representations of music. In addition to numeric event information, it also provides further pitch and time structure information such as contextually correct note names, and key and time signatures.  MusData (http://www.musedata.org), for example, provides access to such a score-based collection.
+Score-based Collections: Score-based data are also symbolic representations of music. In addition to numeric event information, it also provides further pitch and time structure information such as contextually correct note names, and key and time signatures.  MusData (http://www.musedata.org ), for example, provides access to such a score-based collection.
 ==Review 1==

← Older revision		Revision as of 20:44, 1 February 2005
Line 47:		Line 47:

	==Review 2==		==Review 2==
		+
		+	General comment: Title - Evaluation of Key Finding Algorithms Using Symbolic Data or Evaluation of Key Finding Algorithms Part 2
		+
		+	The problem is well defined and the mentioned possible participants seem likely to participate.
		+
		+	Regarding the evaluation procedures, length of input excerpt would have to be determined (15 to 30 seconds - any studies on the ideal length?)
		+	Assumption of closeness:
		+	* Perfect 5th: Is this generally accepted as an almost similar key?
		+	* Parallel major or minor: Not too certain if this needs to be clarified (Ignore this comment if this is generally understood by the majority working in this field)
		+	Based on the error analysis approach outlined, would the algorithm that performs best with the new parameter settings be considered superior ?
		+
		+	The test data are relevant for the problem. For the classical archive web-site, permission would have to be obtained for large downloads at a go, else progressive downloads would have to be worked on (since only a certain number of performances is allowed for downloads in a day).