2007:Audio Cover Song Identification - Revision history

IMIRSELBot: Robot: Automated text replacement (-\[\[([A-Z][^:]+)\]\] +2007:\1)

2010-05-13T18:17:50Z

Robot: Automated text replacement (-\[\[([A-Z][^:]+)\]\] +2007:\1)

JDownie: /* =Audio Format */

2007-08-14T23:31:50Z

‎=Audio Format

JDownie at 23:31, 14 August 2007

2007-08-14T23:31:21Z

JDownie: Audio Cover song identification moved to Audio Cover Song Identification

2007-08-13T21:39:16Z

Audio Cover song identification moved to Audio Cover Song Identification

KrisWest at 17:05, 12 July 2007

2007-07-12T17:05:33Z

Jserra: /* Comments */

2007-07-12T08:01:05Z

‎Comments

Matija: /* Potential Participants */

2007-06-27T07:18:00Z

‎Potential Participants

Jserra: /* Moderators */

2007-06-15T09:39:49Z

‎Moderators

Jserra: /* Comments */

2007-06-14T16:58:54Z

‎Comments

Jserra at 16:50, 14 June 2007

2007-06-14T16:50:53Z

@@ Line 1: / Line 1: @@
 =FINAL 2007 AUDIO COVER SONG IDENTIFICATION TASK OVERVIEW=
-The Audio Cover Song task was a new task for MIREX 2006. It was closely related to the [[Audio Music Similarity and Retrieval]] (AMS) task as the cover songs were embedded in the Audio Music Similarity and Retrieval test collection. However, AMS has change its input format this year so Audio Cover Song and AMS will not be interlinked tasks this year.
+The Audio Cover Song task was a new task for MIREX 2006. It was closely related to the [[2007:Audio Music Similarity and Retrieval]] (AMS) task as the cover songs were embedded in the Audio Music Similarity and Retrieval test collection. However, AMS has change its input format this year so Audio Cover Song and AMS will not be interlinked tasks this year.
 ==Task Description==
@@ Line 95: / Line 95: @@
 ==Moderators==
 * J. Stephen Downie (Principal contact person)
-* Kris West ([[Audio Music Similarity and Retrieval]] moderator)
+* Kris West ([[2007:Audio Music Similarity and Retrieval]] moderator)
 * Dan Ellis
 * Joan Serr├á

← Older revision		Revision as of 23:31, 14 August 2007
Line 6:		Line 6:

	Using each of these cover song files in turn as as the "seed/query" file, we will examine the returned lists of items for the presence of the other 10 versions of the "seed/query" file. See DPWE's Average Precision comments below in the Evaluation discussion section.		Using each of these cover song files in turn as as the "seed/query" file, we will examine the returned lists of items for the presence of the other 10 versions of the "seed/query" file. See DPWE's Average Precision comments below in the Evaluation discussion section.
−
−	~~===Audio Format==~~
−	~~Full length, mono, 16 bit, 22.05kHz, wav~~

	=== Input Files ===		=== Input Files ===

@@ Line 1: / Line 1: @@
-==Overview==
+=FINAL 2007 AUDIO COVER SONG IDENTIFICATION TASK OVERVIEW=
-The Cover Song task was a new task for MIREX 2006. It is closely related to the [[Audio Music Similarity and Retrieval]] task as the cover songs to be found will be embedded in the Audio Music Similarity and Retrieval test collection. Hence, the setup and running of the algorithms will be the same as that of the [[Audio Music Similarity and Retrieval]] so please refer to the [[Audio Music Similarity and Retrieval]] for more details.
+The Audio Cover Song task was a new task for MIREX 2006. It was closely related to the [[Audio Music Similarity and Retrieval]] (AMS) task as the cover songs were embedded in the Audio Music Similarity and Retrieval test collection. However, AMS has change its input format this year so Audio Cover Song and AMS will not be interlinked tasks this year.
 ==Task Description==
-Within the [[Audio Music Similarity and Retrieval]] database, there are embedded 30 different "cover songs" each represented by 11 different "versions" for a total of 330 audio files (16bit, monophonic, 22.05khz, wav). The "cover songs" represent a variety of genres (e.g., classical, jazz, gospel, rock, folk-rock, etc.) and the variations span a variety of styles and orchestrations.
+Within the 1000 pieces in the Audio Cover Song database, there are embedded 30 different "cover songs" each represented by 11 different "versions" for a total of 330 audio files (16bit, monophonic, 22.05khz, wav). The "cover songs" represent a variety of genres (e.g., classical, jazz, gospel, rock, folk-rock, etc.) and the variations span a variety of styles and orchestrations.
-Using each of these cover song files in turn as as the "seed/query" file, we will examine the top 10 returned items for the presence of the other 10 versions of the "seed/query" file.
+Using each of these cover song files in turn as as the "seed/query" file, we will examine the returned lists of items for the presence of the other 10 versions of the "seed/query" file. See DPWE's Average Precision comments below in the Evaluation discussion section.
-Participants in the [[Audio Music Similarity and Retrieval]] task really need not do anything extra as the matrices returned will contain all the necessary information for us to automatically conduct the evaluations. We do encourage, however, the participation of those systems that might be especially designed to detect "cover song" variants.
+===Audio Format==
 ==Audio Format Poll==

← Older revision		Revision as of 17:05, 12 July 2007
Line 8:		Line 8:

	Participants in the [[Audio Music Similarity and Retrieval]] task really need not do anything extra as the matrices returned will contain all the necessary information for us to automatically conduct the evaluations. We do encourage, however, the participation of those systems that might be especially designed to detect "cover song" variants.		Participants in the [[Audio Music Similarity and Retrieval]] task really need not do anything extra as the matrices returned will contain all the necessary information for us to automatically conduct the evaluations. We do encourage, however, the participation of those systems that might be especially designed to detect "cover song" variants.
		+
		+	==Audio Format Poll==
		+
		+	<poll>
		+	Use clips from tracks instead of the whole track?
		+	Yes
		+	No (this changes the task significantly)
		+	No (for some other reason)
		+	</poll>
		+
		+	<poll>
		+	What is your preferred audio format for the Cover song ID task?
		+	22 khz mono WAV
		+	22 khz stereo WAV
		+	44 khz mono WAV
		+	44 khz stereo WAV
		+	22 khz mono MP3 128kb
		+	22 khz stereo MP3 128kb
		+	44 khz mono MP3 128kb
		+	44 khz stereo MP3 128kb
		+	</poll>

	==Evaluation==		==Evaluation==

← Older revision		Revision as of 08:01, 12 July 2007
Line 30:		Line 30:

	- So, my proposal is: GMAP and Geometric mean of the number of correctly classified covers in the 10 first retrieved documents.		- So, my proposal is: GMAP and Geometric mean of the number of correctly classified covers in the 10 first retrieved documents.
		+
		+	--[[User:Jserra\|jserra]] 03:01, 12 July 2007 (CDT)
		+
		+	Audio length: I think one of the most challenging scenarios for a Cover Song Identification system is to deal with different song structures for a group of covers. Therefore, the whole song should be employed for this concrete task instead of the 30s proposed in the Audio Similarity and Retrieval wiki page.

	==Moderators==		==Moderators==

← Older revision	Revision as of 21:39, 13 August 2007
(No difference)

← Older revision		Revision as of 07:18, 27 June 2007
Line 44:		Line 44:
	* Juan P. Bello (jpbello at nyu dot edu)		* Juan P. Bello (jpbello at nyu dot edu)
	* Kyogu Lee (kglee at ccrma dot stanford dot edu)		* Kyogu Lee (kglee at ccrma dot stanford dot edu)
		+	* Matija Marolt (matija.marolt at fri dot uni-lj dot si)

@@ Line 21: / Line 21: @@
 dpwe: In addition to a ranking task, I think it would be interesting to include a detection task i.e. does a cover version of this track exist in this database or not?  This is essentially just setting a threshold on the metric used in ranking the returns, but different algorithms may be better or worse at setting such a threshold that is *consistent* between different tracks.  This could be computed as a Receiver Operating Characteristic i.e. False Alarms vs. False Reject curve as a threshold is varied on the similarity scores returned by the unmodified algorithms.
-jserra:
+--[[User:Jserra|jserra]] 11:58, 14 June 2007 (CDT):
 - I agree on using AP for ranking instead of MRR.