2010:Audio Cover Song Identification Results - Revision history

AndreasEhmann at 20:45, 3 August 2010

2010-08-03T20:45:42Z

Gzhu1 at 18:47, 3 August 2010

2010-08-03T18:47:13Z

MertBay: /* Average Performance per Query Group */

2010-08-03T17:30:08Z

‎Average Performance per Query Group

MertBay: /* Number of Correct Covers at Rank X Returned in Top Ten */

2010-08-03T17:29:57Z

‎Number of Correct Covers at Rank X Returned in Top Ten

MertBay: /* Number of Correct Covers at Rank X Returned in Top Ten */

2010-08-03T17:29:43Z

‎Number of Correct Covers at Rank X Returned in Top Ten

MertBay: /* Results */

2010-08-03T17:29:11Z

‎Results

MertBay at 17:26, 3 August 2010

2010-08-03T17:26:50Z

MertBay at 08:03, 3 August 2010

2010-08-03T08:03:02Z

MertBay: Created page with '== General Legend == {| border="1" cellspacing="0" style="text-align: left; width: 800px;" |- style="background: #efefef;" ! width="80" | Sub code ! width="200" | Submission …'

2010-08-03T06:19:25Z

Created page with '== General Legend == {| border="1" cellspacing="0" style="text-align: left; width: 800px;" |- style="background: #efefef;" ! width="80" | Sub code ! width="200" | Submission …'

New page

== General Legend ==
{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: #efefef;"
! width="80" | Sub code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract
! width="440" | Contributors
|-
! MHRAF1
| Simbals_Cover_Songs || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2010/MHRAF1.pdf PDF] || [http://www.labri.fr Benjamin Martin], [http://simbals.labri.fr Pierre Hanna], [http://simbals.labri.fr Matthias Robine], [http://simbals.labri.fr Julien Allali], [http://simbals.labri.fr Pascal Ferraro]
|-
! MOD1
| Applying Text-Based IR Techniques to Cover Song Identification || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2010/MOD1.pdf PDF] || [http://www.dei.unipd.it/~montecc2 Nicola Montecchio], [http://ims.dei.unipd.it Nicola Orio], [http://www.dei.unipd.it/~dibuccio Emanuele Di Buccio]
|-
! RMHAR1
| Cover || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2010/RMHAR1.pdf PDF] || [http://dept-info.labri.fr/~rocher Thomas Rocher], [http://www.labri.fr Benjamin Martin], [http://dept-info.labri.fr/~hanna Pierre Hanna], [http://simbals.labri.fr Julien Allali], [http://dept-info.labri.fr/~robine Matthias Robine], [http://simbals.labri.fr Pascal Ferraro]
|}

@@ Line 49: / Line 49: @@
 The Friedman test was run in MATLAB against the Average Precision summary data over the 30 song groups.<br /> Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
-<csv p=2>2010//covertests/Coversong_Mixed_Ave_Performance_Per_Query_Group_friedman_tukeyKramerHSD.csv</csv>
+<csv p=2>2010//cover/coversong.mixed.precision.groups.friedman.tukeyKramerHSD.csv</csv>
-[[Image:2010_coversong_mixed_ave_performance_per_query_group_friedman_mean_ranks.png]]
+[[Image:2010coversong.mixed.precision.groups.friedman.tukeyKramerHSD.png|500px]]
 ====Run Times====
@@ Line 71: / Line 71: @@
 The Friedman test was run in MATLAB against the Average Precision summary data over the 30 song groups.<br /> Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
-<csv>2010/covertests/Coversong_Mazurkas_Ave_Performance_Per_Query_Group_friedman_tukeyKramerHSD.csv</csv>
+<csv>2010/cover/coversong.mazurka.precision.groups.friedman.tukeyKramerHSD.csv</csv>
-[[Image:2010_coversong_mazurkas_ave_performance_per_query_group_friedman_mean_ranks.png]]
+[[Image:2010coversong.mazurka.precision.groups.friedman.tukeyKramerHSD.png|500px]]
 ====Run Times====
@@ Line 84: / Line 84: @@
 '''Average Precision by Query'''
-'''RE''' : [https://music-ir.org/mirex/results/2010/coverRE.fix.mix.avgprec.txt S. Ravuri, D.P.W Ellis] <br />
+'''MHRAF1''' : [https://music-ir.org/mirex/results/2010/cover/MHRAF1.1000.avgprec.txt Benjamin Martin et al.] <br />
-'''SZA''' : [https://music-ir.org/mirex/results/2010/coverSZA.mix.avgprec.txt J. Serra et al.]<br />
+'''MOD1''' : [https://music-ir.org/mirex/results/2010/cover/MOD1.1000.avgprec.txt Nicola Montecchio et al.]<br />
-'''TA''' : [https://music-ir.org/mirex/results/2010/coverTA.mix.avgprec.txt T. Ahonen] <br />
+'''RMHAR1''' : [https://music-ir.org/mirex/results/2010/cover/RMHAR1.1000.avgprec.txt Thomas Rocher et al.] <br />
 '''Rank Lists'''
-'''RE''' : [https://music-ir.org/mirex/results/2010/coverRE.fix.mix.ranklist.txt S. Ravuri, D.P.W Ellis] <br />
+'''MHRAF1''' : [https://music-ir.org/mirex/results/2010/cover/MHRAF1.1000.ranklist.txt Benjamin Martin et al.] <br />
-'''SZA''' : [https://music-ir.org/mirex/results/2010/coverSZA.mix.ranklist.txt J. Serra et al.] <br />
+'''MOD1''' : [https://music-ir.org/mirex/results/2010/cover/MOD1.1000.ranklist.txt Nicola Montecchio et al.]<br />
-'''TA''' : [https://music-ir.org/mirex/results/2010/coverTA.mix.ranklist.txt T. Ahonen]<br />
+'''RMHAR1''' : [https://music-ir.org/mirex/results/2010/cover/RMHAR1.1000.ranklist.txt Thomas Rocher et al.] <br />
 ===Sapp's Mazurka Collection===
@@ Line 104: / Line 98: @@
 '''Average Precision by Query'''
-'''RE''' : [https://music-ir.org/mirex/results/2010/coverRE.maz.avgprec.txt S. Ravuri]<br />
+'''MHRAF1''' : [https://music-ir.org/mirex/results/2010/cover/MHRAF1.maz.avgprec.txt Benjamin Martin et al.] <br />
-'''SZA''' : [https://music-ir.org/mirex/results/2010/coverSZA.maz.avgprec.txt J. Serra et al.]<br />
+'''MOD1''' : [https://music-ir.org/mirex/results/2010/cover/MOD1.maz.avgprec.txt Nicola Montecchio et al.]<br />
-'''TA''' : [https://music-ir.org/mirex/results/2010/coverTA.maz.avgprec.txt T. Ahonen]<br />
+'''RMHAR1''' : [https://music-ir.org/mirex/results/2010/cover/RMHAR1.maz.avgprec.txt Thomas Rocher et al.] <br />
 '''Rank Lists'''
-'''RE''' : [https://music-ir.org/mirex/results/2010/coverRE.maz.ranklist.txt S. Ravuri]<br />
+'''MHRAF1''' : [https://music-ir.org/mirex/results/2010/cover/MHRAF1.maz.ranklist.txt Benjamin Martin et al.] <br />
-'''SZA''' : [https://music-ir.org/mirex/results/2010/coverSZA.maz.ranklist.txt J. Serra et al.]<br />
+'''MOD1''' : [https://music-ir.org/mirex/results/2010/cover/MOD1.maz.ranklist.txt Nicola Montecchio et al.]<br />
-'''TA''' : [https://music-ir.org/mirex/results/2010/coverTA.maz.ranklist.txt T. Ahonen]<br />
+'''RMHAR1''' : [https://music-ir.org/mirex/results/2010/cover/RMHAR1.maz.ranklist.txt Thomas Rocher et al.] <br />
 [[Category: Results]]

@@ Line 66: / Line 66: @@
 ====Average Performance per Query Group====
-<csv p=2>2010/covercoversong.mazurka.precision.groups.csv</csv>
+<csv p=2>2010/cover/coversong.mazurka.precision.groups.csv</csv>
 ====Friedman's Test for Significant Differences====

@@ Line 63: / Line 63: @@
 ====Number of Correct Covers at Rank X Returned in Top Ten====
-<csv>2010/covercoversong.mazurka.toptendist.csv</csv>
+<csv>2010/cover/coversong.mazurka.toptendist.csv</csv>
 ====Average Performance per Query Group====

@@ Line 40: / Line 40: @@
 ====Number of Correct Covers at Rank X Returned in Top Ten====
-<csv>2010/cover/rcoversong.mixed.toptendist.csv</csv>
+<csv>2010/cover/coversong.mixed.toptendist.csv</csv>
 ====Average Performance per Query Group====

@@ Line 37: / Line 37: @@
 ====Summary Results====
-<csv p=2>2010//cover/coversong.mixed.summary.csv</csv>
+<csv p=2>2010/cover/coversong.mixed.summary.csv</csv>
 ====Number of Correct Covers at Rank X Returned in Top Ten====
-<csv>2010/covercoversong.mixed.toptendist.csv</csv>
+<csv>2010/cover/rcoversong.mixed.toptendist.csv</csv>
 ====Average Performance per Query Group====
-<csv p=2>2010/covercoversong.mixed.precision.groups.csv</csv>
+<csv p=2>2010/cover/coversong.mixed.precision.groups.csv</csv>

@@ Line 14: / Line 14: @@
 == General Legend ==
 {| border="1" cellspacing="0" style="text-align: left; width: 800px;"
-	|- style="background: #efefef;"
+	|- style="background: yellow;"
 	! width="80" | Sub code
 	! width="200" | Submission name

← Older revision		Revision as of 17:26, 3 August 2010
Line 29:		Line 29:
	\| Cover \|\| style="text-align: center;" \| [https://www.music-ir.org/mirex/abstracts/2010/RMHAR1.pdf PDF] \|\| [http://dept-info.labri.fr/~rocher Thomas Rocher], [http://www.labri.fr Benjamin Martin], [http://dept-info.labri.fr/~hanna Pierre Hanna], [http://simbals.labri.fr Julien Allali], [http://dept-info.labri.fr/~robine Matthias Robine], [http://simbals.labri.fr Pascal Ferraro]		\| Cover \|\| style="text-align: center;" \| [https://www.music-ir.org/mirex/abstracts/2010/RMHAR1.pdf PDF] \|\| [http://dept-info.labri.fr/~rocher Thomas Rocher], [http://www.labri.fr Benjamin Martin], [http://dept-info.labri.fr/~hanna Pierre Hanna], [http://simbals.labri.fr Julien Allali], [http://dept-info.labri.fr/~robine Matthias Robine], [http://simbals.labri.fr Pascal Ferraro]
	\|}		\|}
		+
		+
		+
		+	== Results ==
		+
		+	===Mixed Collection===
		+
		+	====Summary Results====
		+	<csv p=2>2010//cover/coversong.mixed.summary.csv</csv>
		+
		+	====Number of Correct Covers at Rank X Returned in Top Ten====
		+	<csv>2010/covercoversong.mixed.toptendist.csv</csv>
		+
		+	====Average Performance per Query Group====
		+	<csv p=2>2010/covercoversong.mixed.precision.groups.csv</csv>
		+
		+
		+	====Friedman's Test for Significant Differences====
		+	The Friedman test was run in MATLAB against the Average Precision summary data over the 30 song groups.<br /> Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
		+
		+	<csv p=2>2010//covertests/Coversong_Mixed_Ave_Performance_Per_Query_Group_friedman_tukeyKramerHSD.csv</csv>
		+
		+
		+	[[Image:2010_coversong_mixed_ave_performance_per_query_group_friedman_mean_ranks.png]]
		+
		+	====Run Times====
		+	TBA
		+
		+	===Sapp's Mazuraka Collection===
		+
		+	====Summary results====
		+	<csv p=2>2010//cover/coversong.mazurka.summary.csv</csv>
		+
		+	====Number of Correct Covers at Rank X Returned in Top Ten====
		+	<csv>2010/covercoversong.mazurka.toptendist.csv</csv>
		+
		+	====Average Performance per Query Group====
		+	<csv p=2>2010/covercoversong.mazurka.precision.groups.csv</csv>
		+
		+
		+	====Friedman's Test for Significant Differences====
		+	The Friedman test was run in MATLAB against the Average Precision summary data over the 30 song groups.<br /> Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
		+
		+	<csv>2010/covertests/Coversong_Mazurkas_Ave_Performance_Per_Query_Group_friedman_tukeyKramerHSD.csv</csv>
		+
		+
		+	[[Image:2010_coversong_mazurkas_ave_performance_per_query_group_friedman_mean_ranks.png]]
		+
		+	====Run Times====
		+	TBA
		+
		+	==Individual Results Files==
		+
		+	===Mixed Collection===
		+	'''Average Precision by Query'''
		+
		+	'''RE''' : [https://music-ir.org/mirex/results/2010/coverRE.fix.mix.avgprec.txt S. Ravuri, D.P.W Ellis] <br />
		+	'''SZA''' : [https://music-ir.org/mirex/results/2010/coverSZA.mix.avgprec.txt J. Serra et al.]<br />
		+	'''TA''' : [https://music-ir.org/mirex/results/2010/coverTA.mix.avgprec.txt T. Ahonen] <br />
		+
		+	'''Average Precision by Cover Group'''
		+
		+	'''RE''' : [https://music-ir.org/mirex/results/2010/coverRE.fix.mix.avgprec.groups.txt S. Ravuri, D.P.W Ellis]<br />
		+	'''SZA''' : [https://music-ir.org/mirex/results/2010/coverSZA.mix.avgprec.groups.txt J. Serra et al.]<br />
		+	'''TA''' : [https://music-ir.org/mirex/results/2010/coverTA.mix.avgprec.groups.txt T. Ahonen]<br />
		+
		+	'''Rank Lists'''
		+
		+	'''RE''' : [https://music-ir.org/mirex/results/2010/coverRE.fix.mix.ranklist.txt S. Ravuri, D.P.W Ellis] <br />
		+	'''SZA''' : [https://music-ir.org/mirex/results/2010/coverSZA.mix.ranklist.txt J. Serra et al.] <br />
		+	'''TA''' : [https://music-ir.org/mirex/results/2010/coverTA.mix.ranklist.txt T. Ahonen]<br />
		+
		+	===Sapp's Mazurka Collection===
		+
		+	'''Average Precision by Query'''
		+
		+	'''RE''' : [https://music-ir.org/mirex/results/2010/coverRE.maz.avgprec.txt S. Ravuri]<br />
		+	'''SZA''' : [https://music-ir.org/mirex/results/2010/coverSZA.maz.avgprec.txt J. Serra et al.]<br />
		+	'''TA''' : [https://music-ir.org/mirex/results/2010/coverTA.maz.avgprec.txt T. Ahonen]<br />
		+
		+	'''Average Precision by Cover Group'''
		+
		+	'''RE''' : [https://music-ir.org/mirex/results/2010/coverRE.maz.avgprec.groups.txt S. Ravuri]<br />
		+	'''SZA''' : [https://music-ir.org/mirex/results/2010/coverSZA.maz.avgprec.groups.txt J. Serra et al.]<br />
		+	'''TA''' : [https://music-ir.org/mirex/results/2010/coverTA.maz.avgprec.groups.txt T. Ahonen]<br />
		+
		+	'''Rank Lists'''
		+
		+	'''RE''' : [https://music-ir.org/mirex/results/2010/coverRE.maz.ranklist.txt S. Ravuri]<br />
		+	'''SZA''' : [https://music-ir.org/mirex/results/2010/coverSZA.maz.ranklist.txt J. Serra et al.]<br />
		+	'''TA''' : [https://music-ir.org/mirex/results/2010/coverTA.maz.ranklist.txt T. Ahonen]<br />
		+
		+
		+	[[Category: Results]]

← Older revision		Revision as of 08:03, 3 August 2010
Line 1:		Line 1:
		+	== Introduction ==
		+
		+	This year, we ran Audio Cover Song (ACS) Identfication with two datasets: ''Mixed Collection'' and ''Sapp's Mazurka Collection''.
		+
		+	===Mixed Collection Information===
		+	This is the "original" ACS collection. Within the 1000 pieces in the Audio Cover Song database, there are embedded 30 different "cover songs" each represented by 11 different "versions" for a total of 330 audio files (16bit, monophonic, 22.05khz, wav). The "cover songs" represent a variety of genres (e.g., classical, jazz, gospel, rock, folk-rock, etc.) and the variations span a variety of styles and orchestrations.
		+
		+	Using each of these cover song files in turn as as the "seed/query" file, we will examine the returned lists of items for the presence of the other 10 versions of the "seed/query" file.
		+
		+	===Sapp's Mazurka Collection Information ===
		+	In addition to our original ACS dataset, we used the [http://www.mazurka.org.uk/ Mazurka.org dataset] put together by Craig Sapp. We randomly chose 11 versions from 49 mazurkas and ran it as a separate ACS subtask. Systems returned a distance matrix of 539x539 from which we located the ranks of each of the associated cover versions.
		+
		+
	== General Legend ==		== General Legend ==
	{\| border="1" cellspacing="0" style="text-align: left; width: 800px;"		{\| border="1" cellspacing="0" style="text-align: left; width: 800px;"