Difference between revisions of "2009:Audio Music Mood Classification Results"

From MIREX Wiki
(→‎Overall Summary Results)
(→‎Friedman's Test for Significant Differences)
Line 85: Line 85:
 
'''ME3''' = [https://www.music-ir.org/mirex/2008/results/mood/ME3.tar.gz I. M. Mandel, D. P. W. Ellis 3]<br />
 
'''ME3''' = [https://www.music-ir.org/mirex/2008/results/mood/ME3.tar.gz I. M. Mandel, D. P. W. Ellis 3]<br />
  
−
===Friedman's Test for Significant Differences===
+
==Friedman's Tests for Significant Differences==
−
====Classes vs. Systems====
+
===Classes vs. System Tukey-Kramer HSD Multi-Comparisons ===
−
The Friedman test was run in MATLAB against the average accuracy for each class.
+
The Friedman test was run in MATLAB against the average accuracy for each class. The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction. Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);
  
−
=====Friedman's Anova Table=====
+
<csv p=3>audiomood/audiomood_Accuracy.friedman.tukeyKramerHSD.csv</csv>
  
−
<csv>mood/perClassAccuracy.friedman.csv</csv>
+
https://music-ir.org/mirex/2009/results/audiomood/audiomood_Accuracy.friedman.tukeyKramerHSD.png
  
−
=====Tukey-Kramer HSD Multi-Comparison=====
 
−
The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction.
 
−
Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);
 
  
−
<csv>mood/perClassAccuracy.friedman.detail.csv</csv>
+
===Folds vs. Systems Tukey-Kramer HSD Multi-Comparison===
 +
The Friedman test was run in MATLAB against the accuracy for each fold. The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction. Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);
  
−
[[Image:Mood.perClassAccuracy.friedman.tukeyKramerHSD.png]]
+
<csv p=3>audiomood/audiomood_Accuracy_Per_Class.friedman.tukeyKramerHSD.csv
 +
</csv>
  
−
====Folds vs. Systems====
+
https://music-ir.org/mirex/2009/results/audiomood/audiomood_Accuracy_Per_Class.friedman.tukeyKramerHSD.png
−
The Friedman test was run in MATLAB against the accuracy for each fold.
 
−
 
 
−
=====Friedman's Anova Table=====
 
−
 
 
−
<csv>mood/perFoldAccuracy.friedman.csv</csv>
 
−
 
 
−
=====Tukey-Kramer HSD Multi-Comparison=====
 
−
The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction.
 
−
Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);
 
−
 
 
−
<csv>mood/perFoldAccuracy.friedman.detail.csv</csv>
 
−
 
 
−
[[Image:Mood.perFoldAccuracy.friedman.tukeyKramerHSD.png]]
 

Revision as of 20:56, 15 October 2009

Introduction

These are the results for the 2009 running of the Audio Music Mood Classification task. For background information about this task set please refer to the Audio Music Mood Classification page.

General Legend

Team ID

ANO= Anonymous
BP1= [Juan José Burred, Geoffroy Peeters (file)]
BP2 = [Juan José Burred, Geoffroy Peeters (tw)]
CL1 = [Chuan Cao, Ming Li]
CL2 = [Chuan Cao, Ming Li]
FCY1 = [Tao Feng, XiaoOu Chen, DeShun Yang]
FCY2 = [Tao Feng, XiaoOu Chen, DeShun Yang]
GP = [Geoffroy Peeters]
GT1 = [George Tzanetakis (mono)]
GT2 = [George Tzanetakis (stereo)]
GLR1 = [A. Grecu, T. Lidy, A. Rauber (full)]
GLR2 = [A. Grecu, T. Lidy, A. Rauber (template)]
HNOS1 = [Takashi Hasegawa, Takuya Nishimoto, Nobutaka Ono, Shigeki Sagayama (tcca)]
HNOS2 = [Takashi Hasegawa, Takuya Nishimoto, Nobutaka Ono, Shigeki Sagayama (tcck)]
HNOS3 = [Takashi Hasegawa, Takuya Nishimoto, Nobutaka Ono, Shigeki Sagayama (tccl)]
HNOS4 = [Takashi Hasegawa, Takuya Nishimoto, Nobutaka Ono, Shigeki Sagayama (tcpk)]
HW1 = [Huaxin Wang]
HW2 = [Huaxin Wang]
VA1 = [T. Lidy, A. Grecu, A. Rauber, A. Pertusa, P. J. Ponce de Léon, J. M. Iñesta (WMV)]
VA2 = [T. Lidy, A. Grecu, A. Rauber, A. Pertusa, P. J. Ponce de Léon, J. M. Iñesta (BWWV)]
LZG = [Yi Liu, Tao Zheng, Yue Gao (RUC_1)]
RK1 = [Preeti Rao, Sujeet Kini]
RK2 = [Preeti Rao, Sujeet Kini]
RCJ1 = [Jia-Min Ren, Zhi-Sheng Chen, Jyh-Shing Roger Jang]
RCJ2 = [Jia-Min Ren, Zhi-Sheng Chen, Jyh-Shing Roger Jang]
RCJ3 = [Jia-Min Ren, Zhi-Sheng Chen, Jyh-Shing Roger Jang]
RCJ4 = [Jia-Min Ren, Zhi-Sheng Chen, Jyh-Shing Roger Jang]
SS = [Klaus Seyerlehner, Markus Schedl]
TAOS= [Emiru Tsunoo, Taichi Akase, Nobutaka Ono, Shigeki Sagayama]
TTOS = [Emiru Tsunoo, George Tzanetakis, Nobutaka Ono, Shigeki Sagayama]
MTG1 = [N. Wack, E. Guaus, C. Laurier, O. Meyers, R. Marxer, D. Bogdanov, J. Serrà, P. Herrera (false, rca)]
MTG2 = [N. Wack, E. Guaus, C. Laurier, O. Meyers, R. Marxer, D. Bogdanov, J. Serrà, P. Herrera (true, rca)]
MTG3 = [N. Wack, E. Guaus, C. Laurier, O. Meyers, R. Marxer, D. Bogdanov, J. Serrà, P. Herrera (false, simca)]
MTG4 = [N. Wack, E. Guaus, C. Laurier, O. Meyers, R. Marxer, D. Bogdanov, J. Serrà, P. Herrera (true, simca)]
MTG5 = [N. Wack, E. Guaus, C. Laurier, O. Meyers, R. Marxer, D. Bogdanov, J. Serrà, P. Herrera (false, svm)]
MTG6 = [N. Wack, E. Guaus, C. Laurier, O. Meyers, R. Marxer, D. Bogdanov, J. Serrà, P. Herrera (true, svm)]
XLZZG = [Jieping Xu, Yi Liu, Tao Zheng, Chao Zhen, Yue Gao (RUC_1)]
XZZ = [JiePing Xu, Chao Zhen, Tao Zheng (RUC_2)]


Overall Summary Results

MIREX 2009 Audio Mood Classification Summary Results - Raw Classification Accuracy Averaged Over Three Train/Test Folds

file /nema-raid/www/mirex/results/audiomood/summary_audiomood.csv not found

Accuracy Across Folds

file /nema-raid/www/mirex/results/audiomood/audiomood_Accuracy.csv not found

Accuracy Across Categories

file /nema-raid/www/mirex/results/audiomood/audiomood_Accuracy_Per_Class.csv not found

MIREX 2008 Audio Artist Classification Evaluation Logs and Confusion Matrices

MIREX 2008 Audio Mood Classification Run Times

file /nema-raid/www/mirex/results/mood.runtime.csv not found

CSV Files Without Rounding

audiomood_results_fold.csv
audiomood_results_class.csv

Results By Algorithm

(.tar.gz)
GP1 = G. Peeters
GT1 = G. Tzanetakis
GT2 = G. Tzanetakis
GT3 = G. Tzanetakis
HW = G. H. Wang
KL = K. Lee
LRPPI1 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 1
LRPPI2 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 2
LRPPI3 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 3
LRPPI4 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 4
ME1 = I. M. Mandel, D. P. W. Ellis 1
ME2 = I. M. Mandel, D. P. W. Ellis 2
ME3 = I. M. Mandel, D. P. W. Ellis 3

Friedman's Tests for Significant Differences

Classes vs. System Tukey-Kramer HSD Multi-Comparisons

The Friedman test was run in MATLAB against the average accuracy for each class. The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction. Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);

file /nema-raid/www/mirex/results/audiomood/audiomood_Accuracy.friedman.tukeyKramerHSD.csv not found

https://music-ir.org/mirex/2009/results/audiomood/audiomood_Accuracy.friedman.tukeyKramerHSD.png


Folds vs. Systems Tukey-Kramer HSD Multi-Comparison

The Friedman test was run in MATLAB against the accuracy for each fold. The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction. Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);

file /nema-raid/www/mirex/results/audiomood/audiomood_Accuracy_Per_Class.friedman.tukeyKramerHSD.csv not found

https://music-ir.org/mirex/2009/results/audiomood/audiomood_Accuracy_Per_Class.friedman.tukeyKramerHSD.png

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2009:Audio_Music_Mood_Classification_Results&oldid=5632"