Difference between revisions of "2008:Audio Genre Classification Results"

From MIREX Wiki
(→‎Overall Summary Results)
(→‎Overall Summary Results)
Line 85: Line 85:
 
'''ME2''' = [https://www.music-ir.org/mirex/2008/results/genrelatin/ME2.tar.gz I. M. Mandel, D. P. W. Ellis 2]<br />
 
'''ME2''' = [https://www.music-ir.org/mirex/2008/results/genrelatin/ME2.tar.gz I. M. Mandel, D. P. W. Ellis 2]<br />
 
'''ME3''' = [https://www.music-ir.org/mirex/2008/results/genrelatin/ME3.tar.gz I. M. Mandel, D. P. W. Ellis 3]<br />
 
'''ME3''' = [https://www.music-ir.org/mirex/2008/results/genrelatin/ME3.tar.gz I. M. Mandel, D. P. W. Ellis 3]<br />
 +
 +
===Friedman's Test for Significant Differences===
 +
====Task 1 (Mixed) Classes vs. Systems====
 +
The Friedman test was run in MATLAB against the average accuracy for each class.
 +
 +
=====Friedman's Anova Table=====
 +
 +
<csv>genremixed/perClassAccuracy.friedman.csv</csv>
 +
 +
=====Tukey-Kramer HSD Multi-Comparison=====
 +
The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction.
 +
Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);
 +
 +
<csv>Genremixed/perClassAccuracy.friedman.detail.csv</csv>
 +
 +
[[Image:Artist.perClassAccuracy.friedman.tukeyKramerHSD.png]]
 +
 +
====Task 1 (Mixed) Folds vs. Systems====
 +
The Friedman test was run in MATLAB against the accuracy for each fold.
 +
 +
=====Friedman's Anova Table=====
 +
 +
<csv>genremixed/perFoldAccuracy.friedman.csv</csv>
 +
 +
=====Tukey-Kramer HSD Multi-Comparison=====
 +
The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction.
 +
Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);
 +
 +
<csv>genremixed/perFoldAccuracy.friedman.detail.csv</csv>
 +
 +
[[Image:Genremixed.perFoldAccuracy.friedman.tukeyKramerHSD.png]]
 +
 +
  
  
 
[[Category: Results]]
 
[[Category: Results]]

Revision as of 01:49, 11 September 2008

Introduction

These are the results for the 2008 running of the Audio Genre Classification task. For background information about this task set please refer to the Audio Genre Classification page.

General Legend

Team ID

CL1 = C. Cao, M. Li 1
CL2 = C. Cao, M. Li 2
GP1 = G. Peeters
GT1 (mono) = G. Tzanetakis
GT2 (stereo) = G. Tzanetakis
GT3 (multicore) = G. Tzanetakis
LRPPI1 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 1
LRPPI2 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 2
LRPPI3 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 3
LRPPI4 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 4
ME1 = I. M. Mandel, D. P. W. Ellis 1
ME2 = I. M. Mandel, D. P. W. Ellis 2
ME3 = I. M. Mandel, D. P. W. Ellis 3

Overall Summary Results

Task 1 (MIXED) Results

MIREX 2008 Audio Genre Classification Summary Results - Raw and Hierarchical Classification Accuracy Averaged Over Three Train/Test Folds

file /nema-raid/www/mirex/results/genremixed/audiogenre.avg.results.csv not found

Accuracy Across Folds

file /nema-raid/www/mirex/results/genremixed/audiogenre.results.fold.csv not found

Accuracy Across Categories

file /nema-raid/www/mirex/results/genremixed/audiogenre.results.class.csv not found

MIREX 2008 Audio Genre Classification Evaluation Logs and Confusion Matrices

MIREX 2008 Audio Genre Classification Run Times

file /nema-raid/www/mirex/results/genre.runtime.csv not found

CSV Files Without Rounding

audiogenre_results_fold.csv
audiogenre_results_class.csv

Results By Algorithm

(.tar.gz)
CL1 = C. Cao, M. Li 1
CL2 = C. Cao, M. Li 2
LRPPI1 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 1
LRPPI2 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 2
LRPPI3 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 3
LRPPI4 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 4
ME1 = I. M. Mandel, D. P. W. Ellis 1
ME2 = I. M. Mandel, D. P. W. Ellis 2
ME3 = I. M. Mandel, D. P. W. Ellis 3
GP = G. Peeters
GT1 = G. Tzanetakis
GT2 = G. Tzanetakis
GT3 = G. Tzanetakis

Task 2 (LATIN) Results

MIREX 2008 Audio Genre Classification Summary Results - Raw and Hierarchical Classification Accuracy Averaged Over Three Train/Test Folds

file /nema-raid/www/mirex/results/genre.results.csv not found

MIREX 2008 Audio Genre Classification Evaluation Logs and Confusion Matrices

MIREX 2008 Audio Genre Classification Run Times

file /nema-raid/www/mirex/results/latin.runtime.csv not found

Results By Algorithm

(.tar.gz)
CL1 = C. Cao, M. Li 1
CL2 = C. Cao, M. Li 2
GP1 = G. Peeters
GT1 = G. Tzanetakis
GT2 = G. Tzanetakis
GT3 = G. Tzanetakis
LRPPI1 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 1
LRPPI2 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 2
LRPPI3 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 3
LRPPI4 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 4
ME1 = I. M. Mandel, D. P. W. Ellis 1
ME2 = I. M. Mandel, D. P. W. Ellis 2
ME3 = I. M. Mandel, D. P. W. Ellis 3

Friedman's Test for Significant Differences

Task 1 (Mixed) Classes vs. Systems

The Friedman test was run in MATLAB against the average accuracy for each class.

Friedman's Anova Table

file /nema-raid/www/mirex/results/genremixed/perClassAccuracy.friedman.csv not found

Tukey-Kramer HSD Multi-Comparison

The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction. Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);

file /nema-raid/www/mirex/results/Genremixed/perClassAccuracy.friedman.detail.csv not found

File:Artist.perClassAccuracy.friedman.tukeyKramerHSD.png

Task 1 (Mixed) Folds vs. Systems

The Friedman test was run in MATLAB against the accuracy for each fold.

Friedman's Anova Table

file /nema-raid/www/mirex/results/genremixed/perFoldAccuracy.friedman.csv not found

Tukey-Kramer HSD Multi-Comparison

The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction. Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);

file /nema-raid/www/mirex/results/genremixed/perFoldAccuracy.friedman.detail.csv not found

File:Genremixed.perFoldAccuracy.friedman.tukeyKramerHSD.png

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2008:Audio_Genre_Classification_Results&oldid=3991"