Difference between revisions of "2008:Audio Artist Identification Results"

Latest revision as of 16:01, 23 July 2010

Introduction

These are the results for the 2008 running of the Audio Artist Identification task. For background information about this task set please refer to the 2008:Audio Artist Identification page.

General Legend

Team ID

GP2 = G. Peeters
GT1 (mono) = G. Tzanetakis
GT2 (stereo) = G. Tzanetakis
GT3 (multicore) = G. Tzanetakis
LRPPI1 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 1
LRPPI2 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 2
LRPPI3 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 3
LRPPI4 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 4
ME1 = M. I. Mandel, D. P. W. Ellis 1
ME2 = M. I. Mandel, D. P. W. Ellis 2
ME3 = M. I. Mandel, D. P. W. Ellis 3

Overall Summary Results

MIREX 2008 Audio Artist Classification Summary Results - Raw Classification Accuracy Averaged Over Three Train/Test Folds

Participant	Average Classifcation Accuracy
GP2	1.11%
GT1	33.66%
GT2	43.47%
GT3	35.27%
LRPPI1	35.42%
LRPPI2	33.20%
LRPPI3	29.87%
LRPPI4	32.52%
ME1	47.65%
ME2	47.16%
ME3	47.25%

download these results as csv

Accuracy Across Folds

Classification fold	GP2	GT1	GT2	GT3	LRPPI1	LRPPI2	LRPPI3	LRPPI4	ME1	ME2	ME3
0	0.005	0.342	0.443	0.344	0.367	0.348	0.306	0.344	0.467	0.460	0.462
1	0.018	0.329	0.447	0.341	0.333	0.313	0.288	0.308	0.481	0.477	0.475
2	0.011	0.339	0.413	0.373	0.363	0.335	0.303	0.324	0.482	0.478	0.481

download these results as csv

Accuracy Across Categories

Class	GP2	GT1	GT2	GT3	LRPPI1	LRPPI2	LRPPI3	LRPPI4	ME1	ME2	ME3
112	0.000	0.633	0.600	0.433	0.300	0.333	0.300	0.233	0.633	0.633	0.633
ABBA	0.000	0.300	0.233	0.333	0.300	0.167	0.200	0.367	0.500	0.500	0.500
ACDC	0.067	0.433	0.533	0.533	0.533	0.467	0.500	0.467	0.500	0.500	0.500
AEROSMITH	0.100	0.195	0.161	0.195	0.067	0.100	0.100	0.200	0.127	0.127	0.127
ARTHURBIGBOYCRUDUP	0.000	0.933	1.000	1.000	0.900	0.800	0.667	0.733	1.000	1.000	1.000
ARTIESHAW	0.000	0.400	0.700	0.300	0.333	0.400	0.333	0.367	0.433	0.400	0.400
BENNYCARTER	0.000	0.300	0.667	0.367	0.200	0.167	0.133	0.100	0.300	0.300	0.300
BENNYGOODMAN	0.000	0.133	0.367	0.233	0.200	0.233	0.300	0.167	0.367	0.400	0.400
BENNYMOTEN	0.000	0.467	0.700	0.333	0.433	0.267	0.167	0.333	0.433	0.433	0.467
BESSIESMITH	0.033	1.000	1.000	0.900	1.000	0.933	0.967	0.967	1.000	1.000	1.000
BIGBILLBROONZY	0.000	0.600	0.800	0.600	0.533	0.500	0.400	0.500	0.733	0.733	0.733
BILLCOLEMAN	0.000	0.200	0.700	0.133	0.233	0.100	0.133	0.133	0.267	0.267	0.267
BILLIEHOLIDAY	0.000	0.400	0.633	0.333	0.400	0.433	0.300	0.367	0.700	0.700	0.667
BIXBEIDERBECKE	0.000	0.600	0.800	0.600	0.800	0.667	0.600	0.700	0.733	0.733	0.733
BLINK182	0.000	0.167	0.267	0.467	0.200	0.367	0.333	0.333	0.367	0.367	0.367
BOBMARLEYTHEWAILERS	0.000	0.567	0.600	0.633	0.833	0.700	0.733	0.800	0.700	0.700	0.700
BONJOVI	0.000	0.600	0.533	0.567	0.600	0.633	0.633	0.633	0.600	0.533	0.533
BROWNIEMCGHEE	0.000	0.800	0.867	0.833	0.800	0.933	0.700	0.833	0.933	0.933	0.933
BRUCESPRINGSTEEN	0.033	0.067	0.133	0.067	0.367	0.267	0.267	0.300	0.133	0.133	0.133
BRYANADAMS	0.000	0.200	0.400	0.200	0.167	0.133	0.133	0.100	0.367	0.367	0.367
BUNNYBERIGAN	0.000	0.333	0.500	0.433	0.333	0.433	0.433	0.300	0.467	0.433	0.433
CABCALLOWAY	0.000	0.467	0.533	0.500	0.133	0.133	0.133	0.200	0.600	0.600	0.633
CARLPERKINS	0.000	0.567	0.567	0.567	0.400	0.267	0.167	0.300	0.700	0.700	0.700
CHARLIEBARNET	0.000	0.200	0.400	0.300	0.567	0.300	0.333	0.400	0.267	0.267	0.267
CHICKWEBB	0.000	0.267	0.767	0.300	0.300	0.267	0.100	0.167	0.467	0.433	0.433
COUNTBASIE	0.000	0.167	0.300	0.200	0.100	0.133	0.133	0.167	0.333	0.333	0.333
CREEDENCECLEARWATERREVIVAL	0.000	0.500	0.633	0.500	0.533	0.367	0.500	0.533	0.400	0.367	0.367
DAVEMATTHEWSBAND	0.000	0.100	0.133	0.167	0.067	0.133	0.067	0.033	0.367	0.333	0.333
DEANMARTIN	0.000	0.467	0.667	0.567	0.467	0.367	0.367	0.233	0.767	0.767	0.767
DEFLEPPARD	0.000	0.467	0.433	0.667	0.467	0.567	0.467	0.533	0.467	0.467	0.467
DEPECHEMODE	0.000	0.233	0.333	0.200	0.333	0.400	0.167	0.267	0.600	0.600	0.600
DMX	0.067	0.700	0.800	0.700	0.700	0.733	0.733	0.767	0.900	0.900	0.900
DUKEELLINGTON	0.000	0.233	0.400	0.300	0.333	0.333	0.300	0.300	0.300	0.300	0.300
EDDIECONDON	0.000	0.733	0.767	0.733	0.467	0.467	0.533	0.567	0.467	0.467	0.467
ENYA	0.000	0.833	0.833	0.733	0.633	0.733	0.667	0.633	0.667	0.667	0.667
ERICCLAPTON	0.000	0.146	0.305	0.144	0.089	0.057	0.113	0.079	0.144	0.144	0.144
ERSKINEHAWKINS	0.000	0.067	0.100	0.067	0.333	0.333	0.233	0.367	0.333	0.333	0.333
EURYTHMICS	0.000	0.200	0.167	0.200	0.500	0.300	0.400	0.433	0.367	0.367	0.367
FATSWALLER	0.000	0.300	0.667	0.300	0.533	0.433	0.200	0.433	0.700	0.700	0.733
FLEETWOODMAC	0.000	0.000	0.000	0.000	0.033	0.000	0.000	0.033	0.200	0.200	0.200
FLETCHERHENDERSON	0.000	0.100	0.433	0.133	0.167	0.200	0.133	0.100	0.367	0.300	0.300
FOOFIGHTERS	0.300	0.400	0.533	0.667	0.033	0.033	0.133	0.033	0.400	0.400	0.433
FRANKSINATRA	0.000	0.433	0.467	0.400	0.400	0.433	0.467	0.433	0.700	0.667	0.667
FREDDYFENDER	0.000	0.433	0.400	0.433	0.500	0.400	0.200	0.333	0.600	0.533	0.533
GARTHBROOKS	0.000	0.733	0.767	0.633	0.733	0.633	0.533	0.667	0.733	0.733	0.733
GENEKRUPA	0.000	0.333	0.433	0.333	0.200	0.200	0.167	0.233	0.433	0.433	0.467
GENESIS	0.000	0.131	0.083	0.125	0.239	0.175	0.214	0.159	0.042	0.000	0.000
GEORGEJONES	0.000	0.533	0.767	0.567	0.567	0.600	0.600	0.533	0.700	0.700	0.700
GEORGEWETTLING	0.000	0.333	0.533	0.400	0.267	0.100	0.133	0.167	0.233	0.233	0.200
GLENNMILLER	0.000	0.100	0.300	0.200	0.267	0.333	0.133	0.367	0.300	0.300	0.367
GREENDAY	0.000	0.067	0.000	0.100	0.167	0.500	0.433	0.300	0.167	0.167	0.167
HARRYJAMES	0.000	0.267	0.400	0.267	0.233	0.167	0.233	0.167	0.600	0.600	0.600
INXS	0.000	0.033	0.067	0.133	0.133	0.100	0.067	0.100	0.367	0.367	0.367
JACKTEAGARDEN	0.000	0.133	0.233	0.100	0.067	0.167	0.133	0.100	0.400	0.367	0.367
JELLYROLLMORTON	0.000	0.067	0.233	0.100	0.333	0.300	0.233	0.333	0.467	0.467	0.467
JIMMIELUNCEFORD	0.000	0.167	0.267	0.167	0.133	0.300	0.167	0.233	0.400	0.400	0.433
JIMMIENOONE	0.000	0.400	0.733	0.333	0.367	0.200	0.167	0.200	0.467	0.500	0.500
JOHNNYDODDS	0.000	0.100	0.167	0.033	0.200	0.133	0.100	0.133	0.233	0.233	0.233
KOKOMOARNOLD	0.000	0.800	0.867	0.767	0.733	0.867	0.767	0.800	0.867	0.867	0.867
LEADBELLY	0.000	0.533	0.500	0.533	0.367	0.333	0.367	0.333	0.667	0.667	0.667
LIGHTNINHOPKINS	0.000	0.633	0.733	0.667	0.600	0.733	0.733	0.733	0.767	0.767	0.767
LIONELHAMPTON	0.000	0.233	0.433	0.300	0.167	0.333	0.167	0.233	0.467	0.400	0.467
LLCOOLJ	0.000	0.367	0.467	0.400	0.167	0.200	0.167	0.167	0.833	0.833	0.800
LOUISARMSTRONG	0.000	0.000	0.367	0.067	0.233	0.167	0.167	0.167	0.167	0.167	0.167
MADONNA	0.000	0.133	0.067	0.167	0.067	0.033	0.000	0.033	0.367	0.367	0.367
MARIAHCAREY	0.000	0.433	0.600	0.600	0.400	0.400	0.433	0.467	0.800	0.800	0.833
MARILYNMANSON	0.133	0.000	0.067	0.033	0.067	0.100	0.067	0.067	0.367	0.367	0.333
METALLICA	0.000	0.417	0.507	0.424	0.507	0.424	0.444	0.382	0.583	0.583	0.583
MICHAELJACKSON	0.000	0.300	0.267	0.367	0.300	0.267	0.167	0.400	0.267	0.267	0.267
MICKEYGILLEY	0.000	0.567	0.600	0.500	0.667	0.633	0.400	0.467	0.667	0.667	0.633
MUDDYWATERS	0.000	0.500	0.500	0.533	0.333	0.333	0.233	0.433	0.700	0.700	0.700
MUGGSYSPANIER	0.000	0.233	0.267	0.367	0.367	0.300	0.233	0.300	0.533	0.500	0.500
NATKINGCOLE	0.000	0.333	0.633	0.400	0.433	0.400	0.367	0.400	0.700	0.633	0.633
NINEINCHNAILS	0.300	0.033	0.133	0.033	0.067	0.000	0.100	0.000	0.133	0.167	0.100
NIRVANA	0.000	0.167	0.067	0.067	0.233	0.300	0.233	0.233	0.333	0.367	0.367
ORIGINALDIXIELANDJAZZBAND	0.000	1.000	1.000	1.000	0.900	0.733	0.767	0.833	0.900	0.900	0.933
PHILCOLLINS	0.000	0.067	0.100	0.067	0.133	0.233	0.300	0.167	0.433	0.433	0.433
PINKFLOYD	0.000	0.098	0.183	0.278	0.299	0.444	0.156	0.217	0.169	0.169	0.169
QUEEN	0.000	0.067	0.167	0.100	0.267	0.167	0.167	0.200	0.367	0.367	0.367
RADIOHEAD	0.067	0.067	0.067	0.033	0.233	0.167	0.133	0.133	0.033	0.033	0.033
RAYCHARLES	0.000	0.302	0.446	0.346	0.164	0.201	0.204	0.134	0.594	0.594	0.594
REDNICHOLS	0.000	0.367	0.367	0.500	0.367	0.300	0.367	0.400	0.500	0.500	0.500
RKELLY	0.000	0.467	0.433	0.300	0.533	0.400	0.433	0.500	0.533	0.533	0.533
RODSTEWART	0.000	0.067	0.033	0.033	0.167	0.133	0.233	0.100	0.167	0.167	0.167
ROXETTE	0.000	0.167	0.367	0.200	0.167	0.133	0.133	0.133	0.433	0.433	0.433
SADE	0.000	0.133	0.133	0.167	0.133	0.133	0.133	0.133	0.367	0.367	0.367
SANTANA	0.000	0.200	0.133	0.067	0.100	0.067	0.067	0.100	0.133	0.133	0.133
SIDNEYBECHET	0.000	0.300	0.467	0.233	0.300	0.267	0.167	0.100	0.300	0.300	0.300
SONNYBOYWILLIAMSONI	0.000	0.700	0.800	0.733	0.633	0.633	0.633	0.567	0.867	0.867	0.867
STANKENTON	0.000	0.500	0.567	0.533	0.467	0.433	0.300	0.400	0.500	0.467	0.467
SUBLIME	0.000	0.100	0.233	0.233	0.233	0.033	0.167	0.200	0.400	0.400	0.400
THEBEATLES	0.000	0.389	0.167	0.250	0.167	0.222	0.306	0.250	0.278	0.278	0.278
THEDOORS	0.000	0.033	0.067	0.067	0.033	0.000	0.000	0.067	0.233	0.200	0.200
THEROLLINGSTONES	0.028	0.361	0.222	0.278	0.306	0.111	0.083	0.167	0.139	0.139	0.139
TIMMCGRAW	0.000	0.433	0.500	0.300	0.500	0.467	0.400	0.500	0.833	0.867	0.833
TINATURNER	0.000	0.033	0.033	0.095	0.151	0.192	0.167	0.159	0.323	0.323	0.323
TOMJONES	0.000	0.767	0.967	0.833	0.767	0.733	0.633	0.667	0.967	0.967	0.967
TOMMYDORSEY	0.000	0.433	0.433	0.467	0.400	0.300	0.233	0.233	0.433	0.433	0.433
U2	0.000	0.233	0.367	0.233	0.300	0.267	0.133	0.200	0.333	0.333	0.333
VANHALEN	0.000	0.174	0.278	0.174	0.375	0.207	0.269	0.278	0.238	0.238	0.204
WILLIENELSON	0.000	0.667	0.800	0.733	0.733	0.733	0.800	0.867	0.900	0.933	0.933
WOODYHERMAN	0.000	0.033	0.300	0.083	0.167	0.100	0.033	0.033	0.267	0.267	0.267

download these results as csv

MIREX 2008 Audio Artist Classification Evaluation Logs and Confusion Matrices

MIREX 2008 Audio Artist Classification Run Times

Participant	Runtime (hh:mm) / Fold
GP2	Feat Ex: 05:05 Train/Classify: 00:09
GT1	Feat Ex/Train/Classify: 00:16
GT2	Feat Ex/Train/Classify: 00:38
GT3	Feat Ex: 00:06 Train/Classify: 00:00 (11 sec)
LRPPI1	Feat Ex: 13:35 Train/Classify: 00:57
LRPPI2	Feat Ex: 13:35 Train/Classify: 1:15
LRPPI3	Feat Ex: 13:35 Train/Classify: 01:18
LRPPI4	Feat Ex: 13:35 Train/Classify: 01:53
ME1	Feat Ex: 1:27 Train/Classify: 00:01
ME2	Feat Ex: 1:27 Train/Classify: 00:01
ME3	Feat Ex: 1:27 Train/Classify: 00:01

download these results as csv

CSV Files Without Rounding

audioartist_results_fold.csv
audioartist_results_class.csv

Results By Algorithm

(.tar.gz)
GP2 = G. Peeters
GT1 = G. Tzanetakis
GT2 = G. Tzanetakis
GT3 = G. Tzanetakis
LRPPI1 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 1
LRPPI2 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 2
LRPPI3 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 3
LRPPI4 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 4
ME1 = I. M. Mandel, D. P. W. Ellis 1
ME2 = I. M. Mandel, D. P. W. Ellis 2
ME3 = I. M. Mandel, D. P. W. Ellis 3

Friedman's Test for Significant Differences

Classes vs. Systems

The Friedman test was run in MATLAB against the average accuracy for each class.

Friedman's Anova Table

Source	SS	df	MS	Chi-sq	Prob>Chi-sq
Columns	4775.9	10	477.59	452.84	0
Error	5981.6	1010	5.922
Total	10757.5	1121

download these results as csv

Tukey-Kramer HSD Multi-Comparison

The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction. Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
GP2	GT1	-1.2725	0.1912	1.6549	FALSE
GP2	GT2	-1.2431	0.2206	1.6843	FALSE
GP2	GT3	-0.3215	1.1422	2.6058	FALSE
GP2	LRPPI1	1.4481	2.9118	4.3754	TRUE
GP2	LRPPI2	1.1932	2.6569	4.1205	TRUE
GP2	LRPPI3	1.9824	3.4461	4.9098	TRUE
GP2	LRPPI4	1.9285	3.3922	4.8558	TRUE
GP2	ME1	1.9040	3.3676	4.8313	TRUE
GP2	ME2	2.7177	4.1814	5.6450	TRUE
GP2	ME3	5.8206	7.2843	8.7480	TRUE
GT1	GT2	-1.4343	0.0294	1.4931	FALSE
GT1	GT3	-0.5127	0.9510	2.4147	FALSE
GT1	LRPPI1	1.2569	2.7206	4.1843	TRUE
GT1	LRPPI2	1.0020	2.4657	3.9294	TRUE
GT1	LRPPI3	1.7912	3.2549	4.7186	TRUE
GT1	LRPPI4	1.7373	3.2010	4.6647	TRUE
GT1	ME1	1.7128	3.1765	4.6401	TRUE
GT1	ME2	2.5265	3.9902	5.4539	TRUE
GT1	ME3	5.6295	7.0931	8.5568	TRUE
GT2	GT3	-0.5421	0.9216	2.3852	FALSE
GT2	LRPPI1	1.2275	2.6912	4.1549	TRUE
GT2	LRPPI2	0.9726	2.4363	3.8999	TRUE
GT2	LRPPI3	1.7618	3.2255	4.6892	TRUE
GT2	LRPPI4	1.7079	3.1716	4.6352	TRUE
GT2	ME1	1.6834	3.1471	4.6107	TRUE
GT2	ME2	2.4971	3.9608	5.4245	TRUE
GT2	ME3	5.6001	7.0637	8.5274	TRUE
GT3	LRPPI1	0.3059	1.7696	3.2333	TRUE
GT3	LRPPI2	0.0510	1.5147	2.9784	TRUE
GT3	LRPPI3	0.8402	2.3039	3.7676	TRUE
GT3	LRPPI4	0.7863	2.2500	3.7137	TRUE
GT3	ME1	0.7618	2.2255	3.6892	TRUE
GT3	ME2	1.5755	3.0392	4.5029	TRUE
GT3	ME3	4.6785	6.1422	7.6058	TRUE
LRPPI1	LRPPI2	-1.7186	-0.2549	1.2088	FALSE
LRPPI1	LRPPI3	-0.9294	0.5343	1.9980	FALSE
LRPPI1	LRPPI4	-0.9833	0.4804	1.9441	FALSE
LRPPI1	ME1	-1.0078	0.4559	1.9196	FALSE
LRPPI1	ME2	-0.1941	1.2696	2.7333	FALSE
LRPPI1	ME3	2.9089	4.3725	5.8362	TRUE
LRPPI2	LRPPI3	-0.6745	0.7892	2.2529	FALSE
LRPPI2	LRPPI4	-0.7284	0.7353	2.1990	FALSE
LRPPI2	ME1	-0.7529	0.7108	2.1745	FALSE
LRPPI2	ME2	0.0608	1.5245	2.9882	TRUE
LRPPI2	ME3	3.1638	4.6275	6.0911	TRUE
LRPPI3	LRPPI4	-1.5176	-0.0539	1.4098	FALSE
LRPPI3	ME1	-1.5421	-0.0784	1.3852	FALSE
LRPPI3	ME2	-0.7284	0.7353	2.1990	FALSE
LRPPI3	ME3	2.3746	3.8382	5.3019	TRUE
LRPPI4	ME1	-1.4882	-0.0245	1.4392	FALSE
LRPPI4	ME2	-0.6745	0.7892	2.2529	FALSE
LRPPI4	ME3	2.4285	3.8922	5.3558	TRUE
ME1	ME2	-0.6499	0.8137	2.2774	FALSE
ME1	ME3	2.4530	3.9167	5.3803	TRUE
ME2	ME3	1.6393	3.1029	4.5666	TRUE

download these results as csv

Folds vs. Systems

The Friedman test was run in MATLAB against the accuracy for each fold.

Friedman's Anova Table

Source	SS	df	MS	Chi-sq	Prob>Chi-sq
Columns	319.333	10	31.9333	29.03	0.0012
Error	10.667	20	0.5333
Total	330	32

download these results as csv

Tukey-Kramer HSD Multi-Comparison

The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction. Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);

TeamID	TeamID	Lowerbound	Mean	Upperbound	Significance
GP2	GT1	-7.3828	1.3333	10.0495	FALSE
GP2	GT2	-7.0495	1.6667	10.3828	FALSE
GP2	GT3	-5.7162	3.0000	11.7162	FALSE
GP2	LRPPI1	-4.0495	4.6667	13.3828	FALSE
GP2	LRPPI2	-4.0495	4.6667	13.3828	FALSE
GP2	LRPPI3	-2.0495	6.6667	15.3828	FALSE
GP2	LRPPI4	-2.3828	6.3333	15.0495	FALSE
GP2	ME1	-1.0495	7.6667	16.3828	FALSE
GP2	ME2	0.2838	9.0000	17.7162	TRUE
GP2	ME3	1.2838	10.0000	18.7162	TRUE
GT1	GT2	-8.3828	0.3333	9.0495	FALSE
GT1	GT3	-7.0495	1.6667	10.3828	FALSE
GT1	LRPPI1	-5.3828	3.3333	12.0495	FALSE
GT1	LRPPI2	-5.3828	3.3333	12.0495	FALSE
GT1	LRPPI3	-3.3828	5.3333	14.0495	FALSE
GT1	LRPPI4	-3.7162	5.0000	13.7162	FALSE
GT1	ME1	-2.3828	6.3333	15.0495	FALSE
GT1	ME2	-1.0495	7.6667	16.3828	FALSE
GT1	ME3	-0.0495	8.6667	17.3828	FALSE
GT2	GT3	-7.3828	1.3333	10.0495	FALSE
GT2	LRPPI1	-5.7162	3.0000	11.7162	FALSE
GT2	LRPPI2	-5.7162	3.0000	11.7162	FALSE
GT2	LRPPI3	-3.7162	5.0000	13.7162	FALSE
GT2	LRPPI4	-4.0495	4.6667	13.3828	FALSE
GT2	ME1	-2.7162	6.0000	14.7162	FALSE
GT2	ME2	-1.3828	7.3333	16.0495	FALSE
GT2	ME3	-0.3828	8.3333	17.0495	FALSE
GT3	LRPPI1	-7.0495	1.6667	10.3828	FALSE
GT3	LRPPI2	-7.0495	1.6667	10.3828	FALSE
GT3	LRPPI3	-5.0495	3.6667	12.3828	FALSE
GT3	LRPPI4	-5.3828	3.3333	12.0495	FALSE
GT3	ME1	-4.0495	4.6667	13.3828	FALSE
GT3	ME2	-2.7162	6.0000	14.7162	FALSE
GT3	ME3	-1.7162	7.0000	15.7162	FALSE
LRPPI1	LRPPI2	-8.7162	0.0000	8.7162	FALSE
LRPPI1	LRPPI3	-6.7162	2.0000	10.7162	FALSE
LRPPI1	LRPPI4	-7.0495	1.6667	10.3828	FALSE
LRPPI1	ME1	-5.7162	3.0000	11.7162	FALSE
LRPPI1	ME2	-4.3828	4.3333	13.0495	FALSE
LRPPI1	ME3	-3.3828	5.3333	14.0495	FALSE
LRPPI2	LRPPI3	-6.7162	2.0000	10.7162	FALSE
LRPPI2	LRPPI4	-7.0495	1.6667	10.3828	FALSE
LRPPI2	ME1	-5.7162	3.0000	11.7162	FALSE
LRPPI2	ME2	-4.3828	4.3333	13.0495	FALSE
LRPPI2	ME3	-3.3828	5.3333	14.0495	FALSE
LRPPI3	LRPPI4	-9.0495	-0.3333	8.3828	FALSE
LRPPI3	ME1	-7.7162	1.0000	9.7162	FALSE
LRPPI3	ME2	-6.3828	2.3333	11.0495	FALSE
LRPPI3	ME3	-5.3828	3.3333	12.0495	FALSE
LRPPI4	ME1	-7.3828	1.3333	10.0495	FALSE
LRPPI4	ME2	-6.0495	2.6667	11.3828	FALSE
LRPPI4	ME3	-5.0495	3.6667	12.3828	FALSE
ME1	ME2	-7.3828	1.3333	10.0495	FALSE
ME1	ME3	-6.3828	2.3333	11.0495	FALSE
ME2	ME3	-7.7162	1.0000	9.7162	FALSE

download these results as csv

@@ Line 1: / Line 1: @@
 ==Introduction==
-These are the results for the 2008 running of the Audio Artist Identification task. For background information about this task set please refer to the [[Audio Artist Identification]] page.
+These are the results for the 2008 running of the Audio Artist Identification task. For background information about this task set please refer to the [[2008:Audio Artist Identification]] page.
 ===General Legend===
 ====Team ID====
-'''GP2''' = [https://www.music-ir.org/mirex/2008/abs/.pdf G. Peeters]<br />
+'''GP2''' = [https://www.music-ir.org/mirex/abstracts/2008/Peeters_2008_ISMIR_MIREX.pdf G. Peeters]<br />
-'''GT1 (mono)''' = [https://www.music-ir.org/mirex/2008/abs/.pdf G. Tzanetakis]<br />
+'''GT1 (mono)''' = [https://www.music-ir.org/mirex/abstracts/2008/mirex2007.pdf G. Tzanetakis]<br />
-'''GT2 (stereo)''' = [https://www.music-ir.org/mirex/2008/abs/.pdf G. Tzanetakis]<br />
+'''GT2 (stereo)''' = [https://www.music-ir.org/mirex/abstracts/2008/mirex2007.pdf G. Tzanetakis]<br />
-'''GT3 (multicore)''' = [https://www.music-ir.org/mirex/2008/abs/.pdf G. Tzanetakis]<br />
+'''GT3 (multicore)''' = [https://www.music-ir.org/mirex/abstracts/2008/mirex2007.pdf G. Tzanetakis]<br />
-'''LRPPI1''' = [https://www.music-ir.org/mirex/2008/abs/.pdf T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 1]<br />
+'''LRPPI1''' = [https://www.music-ir.org/mirex/abstracts/2008/abstract_mirex08_class.pdf T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 1]<br />
-'''LRPPI2''' = [https://www.music-ir.org/mirex/2008/abs/.pdf T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 2]<br />
+'''LRPPI2''' = [https://www.music-ir.org/mirex/abstracts/2008/abstract_mirex08_class.pdf T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 2]<br />
-'''LRPPI3''' = [https://www.music-ir.org/mirex/2008/abs/.pdf T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 3]<br />
+'''LRPPI3''' = [https://www.music-ir.org/mirex/abstracts/2008/abstract_mirex08_class.pdf T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 3]<br />
-'''LRPPI4''' = [https://www.music-ir.org/mirex/2008/abs/.pdf T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 4]<br />
+'''LRPPI4''' = [https://www.music-ir.org/mirex/abstracts/2008/abstract_mirex08_class.pdf T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 4]<br />
-'''ME1''' = [https://www.music-ir.org/mirex/2008/abs/AI_CC_GC_MC_AS_mandel.pdf M. I. Mandel, D. P. W. Ellis 1]<br />
+'''ME1''' = [https://www.music-ir.org/mirex/abstracts/2008/AA_AG_AT_MM_CC_mandel.pdf M. I. Mandel, D. P. W. Ellis 1]<br />
-'''ME2''' = [https://www.music-ir.org/mirex/2008/abs/AI_CC_GC_MC_AS_mandel.pdf M. I. Mandel, D. P. W. Ellis 2]<br />
+'''ME2''' = [https://www.music-ir.org/mirex/abstracts/2008/AA_AG_AT_MM_CC_mandel.pdf M. I. Mandel, D. P. W. Ellis 2]<br />
-'''ME3''' = [https://www.music-ir.org/mirex/2008/abs/AI_CC_GC_MC_AS_mandel.pdf M. I. Mandel, D. P. W. Ellis 3]<br />
+'''ME3''' = [https://www.music-ir.org/mirex/abstracts/2008/AA_AG_AT_MM_CC_mandel.pdf M. I. Mandel, D. P. W. Ellis 3]<br />
 ==Overall Summary Results==
 ===MIREX 2008 Audio Artist Classification Summary Results - Raw Classification Accuracy Averaged Over Three Train/Test Folds===
-<csv>artist/audioartist.avg.results.csv</csv>
+<csv>2008/artist/audioartist.avg.results.csv</csv>
 =====Accuracy Across Folds=====
-<csv>artist/audioartist.results.fold.csv</csv>
+<csv>2008/artist/audioartist.results.fold.csv</csv>
 =====Accuracy Across Categories=====
-<csv>artist/audioartist.results.class.csv</csv>
+<csv>2008/artist/audioartist.results.class.csv</csv>
 ===MIREX 2008 Audio Artist Classification Evaluation Logs and Confusion Matrices===
@@ Line 33: / Line 33: @@
 ====MIREX 2008 Audio Artist Classification Run Times====
-<csv>artist.runtime.csv</csv>
+<csv>2008/artist.runtime.csv</csv>
 ====CSV Files Without Rounding====
-[https://www.music-ir.org/mirex/2008/results/artist/audioartist_results_fold.csv audioartist_results_fold.csv]<br />
+[https://www.music-ir.org/mirex/results/2008/artist/audioartist_results_fold.csv audioartist_results_fold.csv]<br />
-[https://www.music-ir.org/mirex/2008/results/artist/audioartist_results_class.csv audioartist_results_class.csv]<br />
+[https://www.music-ir.org/mirex/results/2008/artist/audioartist_results_class.csv audioartist_results_class.csv]<br />
 ====Results By Algorithm====
 (.tar.gz) <br />
-'''GP2''' = [https://www.music-ir.org/mirex/2008/results/artist/GP2.tar.gz G. Peeters]<br />
+'''GP2''' = [https://www.music-ir.org/mirex/results/2008/artist/GP2.tar.gz G. Peeters]<br />
-'''GT1''' = [https://www.music-ir.org/mirex/2008/results/artist/GT1.tar.gz G. Tzanetakis]<br />
+'''GT1''' = [https://www.music-ir.org/mirex/results/2008/artist/GT1.tar.gz G. Tzanetakis]<br />
-'''GT2''' = [https://www.music-ir.org/mirex/2008/results/artist/GT2.tar.gz G. Tzanetakis]<br />
+'''GT2''' = [https://www.music-ir.org/mirex/results/2008/artist/GT2.tar.gz G. Tzanetakis]<br />
-'''GT3''' = [https://www.music-ir.org/mirex/2008/results/artist/GT3.tar.gz G. Tzanetakis]<br />
+'''GT3''' = [https://www.music-ir.org/mirex/results/2008/artist/GT3.tar.gz G. Tzanetakis]<br />
-'''LRPPI1''' = [https://www.music-ir.org/mirex/2008/results/artist/LRPPI1.tar.gz T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 1]<br />
+'''LRPPI1''' = [https://www.music-ir.org/mirex/results/2008/artist/LRPPI1.tar.gz T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 1]<br />
-'''LRPPI2''' = [https://www.music-ir.org/mirex/2008/results/artist/LRPPI2.tar.gz T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 2]<br />
+'''LRPPI2''' = [https://www.music-ir.org/mirex/results/2008/artist/LRPPI2.tar.gz T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 2]<br />
-'''LRPPI3''' = [https://www.music-ir.org/mirex/2008/results/artist/LRPPI3.tar.gz T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 3]<br />
+'''LRPPI3''' = [https://www.music-ir.org/mirex/results/2008/artist/LRPPI3.tar.gz T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 3]<br />
-'''LRPPI4''' = [https://www.music-ir.org/mirex/2008/results/artist/LRPPI4.tar.gz T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 4]<br />
+'''LRPPI4''' = [https://www.music-ir.org/mirex/results/2008/artist/LRPPI4.tar.gz T. Lidy, A. Rauber, A. Pertusa, P. Peonce de León, J. M. Iñesta 4]<br />
-'''ME1''' = [https://www.music-ir.org/mirex/2008/results/artist/ME1.tar.gz I. M. Mandel, D. P. W. Ellis 1]<br />
+'''ME1''' = [https://www.music-ir.org/mirex/results/2008/artist/ME1.tar.gz I. M. Mandel, D. P. W. Ellis 1]<br />
-'''ME2''' = [https://www.music-ir.org/mirex/2008/results/artist/ME2.tar.gz I. M. Mandel, D. P. W. Ellis 2]<br />
+'''ME2''' = [https://www.music-ir.org/mirex/results/2008/artist/ME2.tar.gz I. M. Mandel, D. P. W. Ellis 2]<br />
-'''ME3''' = [https://www.music-ir.org/mirex/2008/results/artist/ME3.tar.gz I. M. Mandel, D. P. W. Ellis 3]<br />
+'''ME3''' = [https://www.music-ir.org/mirex/results/2008/artist/ME3.tar.gz I. M. Mandel, D. P. W. Ellis 3]<br />
 ===Friedman's Test for Significant Differences===
@@ Line 59: / Line 59: @@
 =====Friedman's Anova Table=====
-<csv>artist/perClassAccuracy.friedman.csv</csv>
+<csv>2008/artist/perClassAccuracy.friedman.csv</csv>
 =====Tukey-Kramer HSD Multi-Comparison=====
@@ Line 65: / Line 65: @@
 Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);
-<csv>artist/perClassAccuracy.friedman.detail.csv</csv>
+<csv>2008/artist/perClassAccuracy.friedman.detail.csv</csv>
+[[Image:2008_artist.perclassaccuracy.friedman.tukeykramerhsd.png]]
 ====Folds vs. Systems====
@@ Line 72: / Line 74: @@
 =====Friedman's Anova Table=====
-<csv>artist/perFoldAccuracy.friedman.csv</csv>
+<csv>2008/artist/perFoldAccuracy.friedman.csv</csv>
 =====Tukey-Kramer HSD Multi-Comparison=====
@@ Line 78: / Line 80: @@
 Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);
-<csv>artist/perFoldAccuracy.friedman.detail.csv</csv>
+<csv>2008/artist/perFoldAccuracy.friedman.detail.csv</csv>
+[[Image:2008_artist.perfoldaccuracy.friedman.tukeykramerhsd.png]]
 [[Category: Results]]

Difference between revisions of "2008:Audio Artist Identification Results"

Latest revision as of 16:01, 23 July 2010

Contents

Introduction

General Legend

Team ID

Overall Summary Results

MIREX 2008 Audio Artist Classification Summary Results - Raw Classification Accuracy Averaged Over Three Train/Test Folds

Accuracy Across Folds

Accuracy Across Categories

MIREX 2008 Audio Artist Classification Evaluation Logs and Confusion Matrices

MIREX 2008 Audio Artist Classification Run Times

CSV Files Without Rounding

Results By Algorithm

Friedman's Test for Significant Differences

Classes vs. Systems

Friedman's Anova Table

Tukey-Kramer HSD Multi-Comparison

Folds vs. Systems

Friedman's Anova Table

Tukey-Kramer HSD Multi-Comparison

Navigation menu

Views

Personal tools

MIREX by Year

Results by Year

Account Request

Search

Navigation

Tools