Difference between revisions of "2008:Audio Artist Identification Results"
IMIRSELBot (talk | contribs) m (Robot: Automated text replacement (-\[http://www.music-ir.org/mirex/(200[5-9])/abs/ +[http://www.music-ir.org/mirex/abs/\1/)) |
IMIRSELBot (talk | contribs) m (Robot: Automated text replacement (-\[\[Image:\s*([A-Z]) +[[Image:2008_\1)) |
||
| Line 67: | Line 67: | ||
<csv>2008/artist/perClassAccuracy.friedman.detail.csv</csv> | <csv>2008/artist/perClassAccuracy.friedman.detail.csv</csv> | ||
| − | [[Image: | + | [[Image:2008_Artist.perClassAccuracy.friedman.tukeyKramerHSD.png]] |
====Folds vs. Systems==== | ====Folds vs. Systems==== | ||
| Line 82: | Line 82: | ||
<csv>2008/artist/perFoldAccuracy.friedman.detail.csv</csv> | <csv>2008/artist/perFoldAccuracy.friedman.detail.csv</csv> | ||
| − | [[Image: | + | [[Image:2008_Artist.perFoldAccuracy.friedman.tukeyKramerHSD.png]] |
[[Category: Results]] | [[Category: Results]] | ||
Revision as of 22:31, 13 May 2010
Contents
Introduction
These are the results for the 2008 running of the Audio Artist Identification task. For background information about this task set please refer to the 2008:Audio Artist Identification page.
General Legend
Team ID
GP2 = G. Peeters
GT1 (mono) = G. Tzanetakis
GT2 (stereo) = G. Tzanetakis
GT3 (multicore) = G. Tzanetakis
LRPPI1 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 1
LRPPI2 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 2
LRPPI3 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 3
LRPPI4 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 4
ME1 = M. I. Mandel, D. P. W. Ellis 1
ME2 = M. I. Mandel, D. P. W. Ellis 2
ME3 = M. I. Mandel, D. P. W. Ellis 3
Overall Summary Results
MIREX 2008 Audio Artist Classification Summary Results - Raw Classification Accuracy Averaged Over Three Train/Test Folds
| Participant | Average Classifcation Accuracy |
|---|---|
| GP2 | 1.11% |
| GT1 | 33.66% |
| GT2 | 43.47% |
| GT3 | 35.27% |
| LRPPI1 | 35.42% |
| LRPPI2 | 33.20% |
| LRPPI3 | 29.87% |
| LRPPI4 | 32.52% |
| ME1 | 47.65% |
| ME2 | 47.16% |
| ME3 | 47.25% |
Accuracy Across Folds
| Classification fold | GP2 | GT1 | GT2 | GT3 | LRPPI1 | LRPPI2 | LRPPI3 | LRPPI4 | ME1 | ME2 | ME3 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0.005 | 0.342 | 0.443 | 0.344 | 0.367 | 0.348 | 0.306 | 0.344 | 0.467 | 0.460 | 0.462 |
| 1 | 0.018 | 0.329 | 0.447 | 0.341 | 0.333 | 0.313 | 0.288 | 0.308 | 0.481 | 0.477 | 0.475 |
| 2 | 0.011 | 0.339 | 0.413 | 0.373 | 0.363 | 0.335 | 0.303 | 0.324 | 0.482 | 0.478 | 0.481 |
Accuracy Across Categories
| Class | GP2 | GT1 | GT2 | GT3 | LRPPI1 | LRPPI2 | LRPPI3 | LRPPI4 | ME1 | ME2 | ME3 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 112 | 0.000 | 0.633 | 0.600 | 0.433 | 0.300 | 0.333 | 0.300 | 0.233 | 0.633 | 0.633 | 0.633 |
| ABBA | 0.000 | 0.300 | 0.233 | 0.333 | 0.300 | 0.167 | 0.200 | 0.367 | 0.500 | 0.500 | 0.500 |
| ACDC | 0.067 | 0.433 | 0.533 | 0.533 | 0.533 | 0.467 | 0.500 | 0.467 | 0.500 | 0.500 | 0.500 |
| AEROSMITH | 0.100 | 0.195 | 0.161 | 0.195 | 0.067 | 0.100 | 0.100 | 0.200 | 0.127 | 0.127 | 0.127 |
| ARTHURBIGBOYCRUDUP | 0.000 | 0.933 | 1.000 | 1.000 | 0.900 | 0.800 | 0.667 | 0.733 | 1.000 | 1.000 | 1.000 |
| ARTIESHAW | 0.000 | 0.400 | 0.700 | 0.300 | 0.333 | 0.400 | 0.333 | 0.367 | 0.433 | 0.400 | 0.400 |
| BENNYCARTER | 0.000 | 0.300 | 0.667 | 0.367 | 0.200 | 0.167 | 0.133 | 0.100 | 0.300 | 0.300 | 0.300 |
| BENNYGOODMAN | 0.000 | 0.133 | 0.367 | 0.233 | 0.200 | 0.233 | 0.300 | 0.167 | 0.367 | 0.400 | 0.400 |
| BENNYMOTEN | 0.000 | 0.467 | 0.700 | 0.333 | 0.433 | 0.267 | 0.167 | 0.333 | 0.433 | 0.433 | 0.467 |
| BESSIESMITH | 0.033 | 1.000 | 1.000 | 0.900 | 1.000 | 0.933 | 0.967 | 0.967 | 1.000 | 1.000 | 1.000 |
| BIGBILLBROONZY | 0.000 | 0.600 | 0.800 | 0.600 | 0.533 | 0.500 | 0.400 | 0.500 | 0.733 | 0.733 | 0.733 |
| BILLCOLEMAN | 0.000 | 0.200 | 0.700 | 0.133 | 0.233 | 0.100 | 0.133 | 0.133 | 0.267 | 0.267 | 0.267 |
| BILLIEHOLIDAY | 0.000 | 0.400 | 0.633 | 0.333 | 0.400 | 0.433 | 0.300 | 0.367 | 0.700 | 0.700 | 0.667 |
| BIXBEIDERBECKE | 0.000 | 0.600 | 0.800 | 0.600 | 0.800 | 0.667 | 0.600 | 0.700 | 0.733 | 0.733 | 0.733 |
| BLINK182 | 0.000 | 0.167 | 0.267 | 0.467 | 0.200 | 0.367 | 0.333 | 0.333 | 0.367 | 0.367 | 0.367 |
| BOBMARLEYTHEWAILERS | 0.000 | 0.567 | 0.600 | 0.633 | 0.833 | 0.700 | 0.733 | 0.800 | 0.700 | 0.700 | 0.700 |
| BONJOVI | 0.000 | 0.600 | 0.533 | 0.567 | 0.600 | 0.633 | 0.633 | 0.633 | 0.600 | 0.533 | 0.533 |
| BROWNIEMCGHEE | 0.000 | 0.800 | 0.867 | 0.833 | 0.800 | 0.933 | 0.700 | 0.833 | 0.933 | 0.933 | 0.933 |
| BRUCESPRINGSTEEN | 0.033 | 0.067 | 0.133 | 0.067 | 0.367 | 0.267 | 0.267 | 0.300 | 0.133 | 0.133 | 0.133 |
| BRYANADAMS | 0.000 | 0.200 | 0.400 | 0.200 | 0.167 | 0.133 | 0.133 | 0.100 | 0.367 | 0.367 | 0.367 |
| BUNNYBERIGAN | 0.000 | 0.333 | 0.500 | 0.433 | 0.333 | 0.433 | 0.433 | 0.300 | 0.467 | 0.433 | 0.433 |
| CABCALLOWAY | 0.000 | 0.467 | 0.533 | 0.500 | 0.133 | 0.133 | 0.133 | 0.200 | 0.600 | 0.600 | 0.633 |
| CARLPERKINS | 0.000 | 0.567 | 0.567 | 0.567 | 0.400 | 0.267 | 0.167 | 0.300 | 0.700 | 0.700 | 0.700 |
| CHARLIEBARNET | 0.000 | 0.200 | 0.400 | 0.300 | 0.567 | 0.300 | 0.333 | 0.400 | 0.267 | 0.267 | 0.267 |
| CHICKWEBB | 0.000 | 0.267 | 0.767 | 0.300 | 0.300 | 0.267 | 0.100 | 0.167 | 0.467 | 0.433 | 0.433 |
| COUNTBASIE | 0.000 | 0.167 | 0.300 | 0.200 | 0.100 | 0.133 | 0.133 | 0.167 | 0.333 | 0.333 | 0.333 |
| CREEDENCECLEARWATERREVIVAL | 0.000 | 0.500 | 0.633 | 0.500 | 0.533 | 0.367 | 0.500 | 0.533 | 0.400 | 0.367 | 0.367 |
| DAVEMATTHEWSBAND | 0.000 | 0.100 | 0.133 | 0.167 | 0.067 | 0.133 | 0.067 | 0.033 | 0.367 | 0.333 | 0.333 |
| DEANMARTIN | 0.000 | 0.467 | 0.667 | 0.567 | 0.467 | 0.367 | 0.367 | 0.233 | 0.767 | 0.767 | 0.767 |
| DEFLEPPARD | 0.000 | 0.467 | 0.433 | 0.667 | 0.467 | 0.567 | 0.467 | 0.533 | 0.467 | 0.467 | 0.467 |
| DEPECHEMODE | 0.000 | 0.233 | 0.333 | 0.200 | 0.333 | 0.400 | 0.167 | 0.267 | 0.600 | 0.600 | 0.600 |
| DMX | 0.067 | 0.700 | 0.800 | 0.700 | 0.700 | 0.733 | 0.733 | 0.767 | 0.900 | 0.900 | 0.900 |
| DUKEELLINGTON | 0.000 | 0.233 | 0.400 | 0.300 | 0.333 | 0.333 | 0.300 | 0.300 | 0.300 | 0.300 | 0.300 |
| EDDIECONDON | 0.000 | 0.733 | 0.767 | 0.733 | 0.467 | 0.467 | 0.533 | 0.567 | 0.467 | 0.467 | 0.467 |
| ENYA | 0.000 | 0.833 | 0.833 | 0.733 | 0.633 | 0.733 | 0.667 | 0.633 | 0.667 | 0.667 | 0.667 |
| ERICCLAPTON | 0.000 | 0.146 | 0.305 | 0.144 | 0.089 | 0.057 | 0.113 | 0.079 | 0.144 | 0.144 | 0.144 |
| ERSKINEHAWKINS | 0.000 | 0.067 | 0.100 | 0.067 | 0.333 | 0.333 | 0.233 | 0.367 | 0.333 | 0.333 | 0.333 |
| EURYTHMICS | 0.000 | 0.200 | 0.167 | 0.200 | 0.500 | 0.300 | 0.400 | 0.433 | 0.367 | 0.367 | 0.367 |
| FATSWALLER | 0.000 | 0.300 | 0.667 | 0.300 | 0.533 | 0.433 | 0.200 | 0.433 | 0.700 | 0.700 | 0.733 |
| FLEETWOODMAC | 0.000 | 0.000 | 0.000 | 0.000 | 0.033 | 0.000 | 0.000 | 0.033 | 0.200 | 0.200 | 0.200 |
| FLETCHERHENDERSON | 0.000 | 0.100 | 0.433 | 0.133 | 0.167 | 0.200 | 0.133 | 0.100 | 0.367 | 0.300 | 0.300 |
| FOOFIGHTERS | 0.300 | 0.400 | 0.533 | 0.667 | 0.033 | 0.033 | 0.133 | 0.033 | 0.400 | 0.400 | 0.433 |
| FRANKSINATRA | 0.000 | 0.433 | 0.467 | 0.400 | 0.400 | 0.433 | 0.467 | 0.433 | 0.700 | 0.667 | 0.667 |
| FREDDYFENDER | 0.000 | 0.433 | 0.400 | 0.433 | 0.500 | 0.400 | 0.200 | 0.333 | 0.600 | 0.533 | 0.533 |
| GARTHBROOKS | 0.000 | 0.733 | 0.767 | 0.633 | 0.733 | 0.633 | 0.533 | 0.667 | 0.733 | 0.733 | 0.733 |
| GENEKRUPA | 0.000 | 0.333 | 0.433 | 0.333 | 0.200 | 0.200 | 0.167 | 0.233 | 0.433 | 0.433 | 0.467 |
| GENESIS | 0.000 | 0.131 | 0.083 | 0.125 | 0.239 | 0.175 | 0.214 | 0.159 | 0.042 | 0.000 | 0.000 |
| GEORGEJONES | 0.000 | 0.533 | 0.767 | 0.567 | 0.567 | 0.600 | 0.600 | 0.533 | 0.700 | 0.700 | 0.700 |
| GEORGEWETTLING | 0.000 | 0.333 | 0.533 | 0.400 | 0.267 | 0.100 | 0.133 | 0.167 | 0.233 | 0.233 | 0.200 |
| GLENNMILLER | 0.000 | 0.100 | 0.300 | 0.200 | 0.267 | 0.333 | 0.133 | 0.367 | 0.300 | 0.300 | 0.367 |
| GREENDAY | 0.000 | 0.067 | 0.000 | 0.100 | 0.167 | 0.500 | 0.433 | 0.300 | 0.167 | 0.167 | 0.167 |
| HARRYJAMES | 0.000 | 0.267 | 0.400 | 0.267 | 0.233 | 0.167 | 0.233 | 0.167 | 0.600 | 0.600 | 0.600 |
| INXS | 0.000 | 0.033 | 0.067 | 0.133 | 0.133 | 0.100 | 0.067 | 0.100 | 0.367 | 0.367 | 0.367 |
| JACKTEAGARDEN | 0.000 | 0.133 | 0.233 | 0.100 | 0.067 | 0.167 | 0.133 | 0.100 | 0.400 | 0.367 | 0.367 |
| JELLYROLLMORTON | 0.000 | 0.067 | 0.233 | 0.100 | 0.333 | 0.300 | 0.233 | 0.333 | 0.467 | 0.467 | 0.467 |
| JIMMIELUNCEFORD | 0.000 | 0.167 | 0.267 | 0.167 | 0.133 | 0.300 | 0.167 | 0.233 | 0.400 | 0.400 | 0.433 |
| JIMMIENOONE | 0.000 | 0.400 | 0.733 | 0.333 | 0.367 | 0.200 | 0.167 | 0.200 | 0.467 | 0.500 | 0.500 |
| JOHNNYDODDS | 0.000 | 0.100 | 0.167 | 0.033 | 0.200 | 0.133 | 0.100 | 0.133 | 0.233 | 0.233 | 0.233 |
| KOKOMOARNOLD | 0.000 | 0.800 | 0.867 | 0.767 | 0.733 | 0.867 | 0.767 | 0.800 | 0.867 | 0.867 | 0.867 |
| LEADBELLY | 0.000 | 0.533 | 0.500 | 0.533 | 0.367 | 0.333 | 0.367 | 0.333 | 0.667 | 0.667 | 0.667 |
| LIGHTNINHOPKINS | 0.000 | 0.633 | 0.733 | 0.667 | 0.600 | 0.733 | 0.733 | 0.733 | 0.767 | 0.767 | 0.767 |
| LIONELHAMPTON | 0.000 | 0.233 | 0.433 | 0.300 | 0.167 | 0.333 | 0.167 | 0.233 | 0.467 | 0.400 | 0.467 |
| LLCOOLJ | 0.000 | 0.367 | 0.467 | 0.400 | 0.167 | 0.200 | 0.167 | 0.167 | 0.833 | 0.833 | 0.800 |
| LOUISARMSTRONG | 0.000 | 0.000 | 0.367 | 0.067 | 0.233 | 0.167 | 0.167 | 0.167 | 0.167 | 0.167 | 0.167 |
| MADONNA | 0.000 | 0.133 | 0.067 | 0.167 | 0.067 | 0.033 | 0.000 | 0.033 | 0.367 | 0.367 | 0.367 |
| MARIAHCAREY | 0.000 | 0.433 | 0.600 | 0.600 | 0.400 | 0.400 | 0.433 | 0.467 | 0.800 | 0.800 | 0.833 |
| MARILYNMANSON | 0.133 | 0.000 | 0.067 | 0.033 | 0.067 | 0.100 | 0.067 | 0.067 | 0.367 | 0.367 | 0.333 |
| METALLICA | 0.000 | 0.417 | 0.507 | 0.424 | 0.507 | 0.424 | 0.444 | 0.382 | 0.583 | 0.583 | 0.583 |
| MICHAELJACKSON | 0.000 | 0.300 | 0.267 | 0.367 | 0.300 | 0.267 | 0.167 | 0.400 | 0.267 | 0.267 | 0.267 |
| MICKEYGILLEY | 0.000 | 0.567 | 0.600 | 0.500 | 0.667 | 0.633 | 0.400 | 0.467 | 0.667 | 0.667 | 0.633 |
| MUDDYWATERS | 0.000 | 0.500 | 0.500 | 0.533 | 0.333 | 0.333 | 0.233 | 0.433 | 0.700 | 0.700 | 0.700 |
| MUGGSYSPANIER | 0.000 | 0.233 | 0.267 | 0.367 | 0.367 | 0.300 | 0.233 | 0.300 | 0.533 | 0.500 | 0.500 |
| NATKINGCOLE | 0.000 | 0.333 | 0.633 | 0.400 | 0.433 | 0.400 | 0.367 | 0.400 | 0.700 | 0.633 | 0.633 |
| NINEINCHNAILS | 0.300 | 0.033 | 0.133 | 0.033 | 0.067 | 0.000 | 0.100 | 0.000 | 0.133 | 0.167 | 0.100 |
| NIRVANA | 0.000 | 0.167 | 0.067 | 0.067 | 0.233 | 0.300 | 0.233 | 0.233 | 0.333 | 0.367 | 0.367 |
| ORIGINALDIXIELANDJAZZBAND | 0.000 | 1.000 | 1.000 | 1.000 | 0.900 | 0.733 | 0.767 | 0.833 | 0.900 | 0.900 | 0.933 |
| PHILCOLLINS | 0.000 | 0.067 | 0.100 | 0.067 | 0.133 | 0.233 | 0.300 | 0.167 | 0.433 | 0.433 | 0.433 |
| PINKFLOYD | 0.000 | 0.098 | 0.183 | 0.278 | 0.299 | 0.444 | 0.156 | 0.217 | 0.169 | 0.169 | 0.169 |
| QUEEN | 0.000 | 0.067 | 0.167 | 0.100 | 0.267 | 0.167 | 0.167 | 0.200 | 0.367 | 0.367 | 0.367 |
| RADIOHEAD | 0.067 | 0.067 | 0.067 | 0.033 | 0.233 | 0.167 | 0.133 | 0.133 | 0.033 | 0.033 | 0.033 |
| RAYCHARLES | 0.000 | 0.302 | 0.446 | 0.346 | 0.164 | 0.201 | 0.204 | 0.134 | 0.594 | 0.594 | 0.594 |
| REDNICHOLS | 0.000 | 0.367 | 0.367 | 0.500 | 0.367 | 0.300 | 0.367 | 0.400 | 0.500 | 0.500 | 0.500 |
| RKELLY | 0.000 | 0.467 | 0.433 | 0.300 | 0.533 | 0.400 | 0.433 | 0.500 | 0.533 | 0.533 | 0.533 |
| RODSTEWART | 0.000 | 0.067 | 0.033 | 0.033 | 0.167 | 0.133 | 0.233 | 0.100 | 0.167 | 0.167 | 0.167 |
| ROXETTE | 0.000 | 0.167 | 0.367 | 0.200 | 0.167 | 0.133 | 0.133 | 0.133 | 0.433 | 0.433 | 0.433 |
| SADE | 0.000 | 0.133 | 0.133 | 0.167 | 0.133 | 0.133 | 0.133 | 0.133 | 0.367 | 0.367 | 0.367 |
| SANTANA | 0.000 | 0.200 | 0.133 | 0.067 | 0.100 | 0.067 | 0.067 | 0.100 | 0.133 | 0.133 | 0.133 |
| SIDNEYBECHET | 0.000 | 0.300 | 0.467 | 0.233 | 0.300 | 0.267 | 0.167 | 0.100 | 0.300 | 0.300 | 0.300 |
| SONNYBOYWILLIAMSONI | 0.000 | 0.700 | 0.800 | 0.733 | 0.633 | 0.633 | 0.633 | 0.567 | 0.867 | 0.867 | 0.867 |
| STANKENTON | 0.000 | 0.500 | 0.567 | 0.533 | 0.467 | 0.433 | 0.300 | 0.400 | 0.500 | 0.467 | 0.467 |
| SUBLIME | 0.000 | 0.100 | 0.233 | 0.233 | 0.233 | 0.033 | 0.167 | 0.200 | 0.400 | 0.400 | 0.400 |
| THEBEATLES | 0.000 | 0.389 | 0.167 | 0.250 | 0.167 | 0.222 | 0.306 | 0.250 | 0.278 | 0.278 | 0.278 |
| THEDOORS | 0.000 | 0.033 | 0.067 | 0.067 | 0.033 | 0.000 | 0.000 | 0.067 | 0.233 | 0.200 | 0.200 |
| THEROLLINGSTONES | 0.028 | 0.361 | 0.222 | 0.278 | 0.306 | 0.111 | 0.083 | 0.167 | 0.139 | 0.139 | 0.139 |
| TIMMCGRAW | 0.000 | 0.433 | 0.500 | 0.300 | 0.500 | 0.467 | 0.400 | 0.500 | 0.833 | 0.867 | 0.833 |
| TINATURNER | 0.000 | 0.033 | 0.033 | 0.095 | 0.151 | 0.192 | 0.167 | 0.159 | 0.323 | 0.323 | 0.323 |
| TOMJONES | 0.000 | 0.767 | 0.967 | 0.833 | 0.767 | 0.733 | 0.633 | 0.667 | 0.967 | 0.967 | 0.967 |
| TOMMYDORSEY | 0.000 | 0.433 | 0.433 | 0.467 | 0.400 | 0.300 | 0.233 | 0.233 | 0.433 | 0.433 | 0.433 |
| U2 | 0.000 | 0.233 | 0.367 | 0.233 | 0.300 | 0.267 | 0.133 | 0.200 | 0.333 | 0.333 | 0.333 |
| VANHALEN | 0.000 | 0.174 | 0.278 | 0.174 | 0.375 | 0.207 | 0.269 | 0.278 | 0.238 | 0.238 | 0.204 |
| WILLIENELSON | 0.000 | 0.667 | 0.800 | 0.733 | 0.733 | 0.733 | 0.800 | 0.867 | 0.900 | 0.933 | 0.933 |
| WOODYHERMAN | 0.000 | 0.033 | 0.300 | 0.083 | 0.167 | 0.100 | 0.033 | 0.033 | 0.267 | 0.267 | 0.267 |
MIREX 2008 Audio Artist Classification Evaluation Logs and Confusion Matrices
MIREX 2008 Audio Artist Classification Run Times
| Participant | Runtime (hh:mm) / Fold |
|---|---|
| GP2 | Feat Ex: 05:05 Train/Classify: 00:09 |
| GT1 | Feat Ex/Train/Classify: 00:16 |
| GT2 | Feat Ex/Train/Classify: 00:38 |
| GT3 | Feat Ex: 00:06 Train/Classify: 00:00 (11 sec) |
| LRPPI1 | Feat Ex: 13:35 Train/Classify: 00:57 |
| LRPPI2 | Feat Ex: 13:35 Train/Classify: 1:15 |
| LRPPI3 | Feat Ex: 13:35 Train/Classify: 01:18 |
| LRPPI4 | Feat Ex: 13:35 Train/Classify: 01:53 |
| ME1 | Feat Ex: 1:27 Train/Classify: 00:01 |
| ME2 | Feat Ex: 1:27 Train/Classify: 00:01 |
| ME3 | Feat Ex: 1:27 Train/Classify: 00:01 |
CSV Files Without Rounding
audioartist_results_fold.csv
audioartist_results_class.csv
Results By Algorithm
(.tar.gz)
GP2 = G. Peeters
GT1 = G. Tzanetakis
GT2 = G. Tzanetakis
GT3 = G. Tzanetakis
LRPPI1 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 1
LRPPI2 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 2
LRPPI3 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 3
LRPPI4 = T. Lidy, A. Rauber, A. Pertusa, P. Peonce de Leon, J. M. I├▒esta 4
ME1 = I. M. Mandel, D. P. W. Ellis 1
ME2 = I. M. Mandel, D. P. W. Ellis 2
ME3 = I. M. Mandel, D. P. W. Ellis 3
Friedman's Test for Significant Differences
Classes vs. Systems
The Friedman test was run in MATLAB against the average accuracy for each class.
Friedman's Anova Table
| Source | SS | df | MS | Chi-sq | Prob>Chi-sq |
|---|---|---|---|---|---|
| Columns | 4775.9 | 10 | 477.59 | 452.84 | 0 |
| Error | 5981.6 | 1010 | 5.922 | ||
| Total | 10757.5 | 1121 |
Tukey-Kramer HSD Multi-Comparison
The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction. Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);
| TeamID | TeamID | Lowerbound | Mean | Upperbound | Significance |
|---|---|---|---|---|---|
| GP2 | GT1 | -1.2725 | 0.1912 | 1.6549 | FALSE |
| GP2 | GT2 | -1.2431 | 0.2206 | 1.6843 | FALSE |
| GP2 | GT3 | -0.3215 | 1.1422 | 2.6058 | FALSE |
| GP2 | LRPPI1 | 1.4481 | 2.9118 | 4.3754 | TRUE |
| GP2 | LRPPI2 | 1.1932 | 2.6569 | 4.1205 | TRUE |
| GP2 | LRPPI3 | 1.9824 | 3.4461 | 4.9098 | TRUE |
| GP2 | LRPPI4 | 1.9285 | 3.3922 | 4.8558 | TRUE |
| GP2 | ME1 | 1.9040 | 3.3676 | 4.8313 | TRUE |
| GP2 | ME2 | 2.7177 | 4.1814 | 5.6450 | TRUE |
| GP2 | ME3 | 5.8206 | 7.2843 | 8.7480 | TRUE |
| GT1 | GT2 | -1.4343 | 0.0294 | 1.4931 | FALSE |
| GT1 | GT3 | -0.5127 | 0.9510 | 2.4147 | FALSE |
| GT1 | LRPPI1 | 1.2569 | 2.7206 | 4.1843 | TRUE |
| GT1 | LRPPI2 | 1.0020 | 2.4657 | 3.9294 | TRUE |
| GT1 | LRPPI3 | 1.7912 | 3.2549 | 4.7186 | TRUE |
| GT1 | LRPPI4 | 1.7373 | 3.2010 | 4.6647 | TRUE |
| GT1 | ME1 | 1.7128 | 3.1765 | 4.6401 | TRUE |
| GT1 | ME2 | 2.5265 | 3.9902 | 5.4539 | TRUE |
| GT1 | ME3 | 5.6295 | 7.0931 | 8.5568 | TRUE |
| GT2 | GT3 | -0.5421 | 0.9216 | 2.3852 | FALSE |
| GT2 | LRPPI1 | 1.2275 | 2.6912 | 4.1549 | TRUE |
| GT2 | LRPPI2 | 0.9726 | 2.4363 | 3.8999 | TRUE |
| GT2 | LRPPI3 | 1.7618 | 3.2255 | 4.6892 | TRUE |
| GT2 | LRPPI4 | 1.7079 | 3.1716 | 4.6352 | TRUE |
| GT2 | ME1 | 1.6834 | 3.1471 | 4.6107 | TRUE |
| GT2 | ME2 | 2.4971 | 3.9608 | 5.4245 | TRUE |
| GT2 | ME3 | 5.6001 | 7.0637 | 8.5274 | TRUE |
| GT3 | LRPPI1 | 0.3059 | 1.7696 | 3.2333 | TRUE |
| GT3 | LRPPI2 | 0.0510 | 1.5147 | 2.9784 | TRUE |
| GT3 | LRPPI3 | 0.8402 | 2.3039 | 3.7676 | TRUE |
| GT3 | LRPPI4 | 0.7863 | 2.2500 | 3.7137 | TRUE |
| GT3 | ME1 | 0.7618 | 2.2255 | 3.6892 | TRUE |
| GT3 | ME2 | 1.5755 | 3.0392 | 4.5029 | TRUE |
| GT3 | ME3 | 4.6785 | 6.1422 | 7.6058 | TRUE |
| LRPPI1 | LRPPI2 | -1.7186 | -0.2549 | 1.2088 | FALSE |
| LRPPI1 | LRPPI3 | -0.9294 | 0.5343 | 1.9980 | FALSE |
| LRPPI1 | LRPPI4 | -0.9833 | 0.4804 | 1.9441 | FALSE |
| LRPPI1 | ME1 | -1.0078 | 0.4559 | 1.9196 | FALSE |
| LRPPI1 | ME2 | -0.1941 | 1.2696 | 2.7333 | FALSE |
| LRPPI1 | ME3 | 2.9089 | 4.3725 | 5.8362 | TRUE |
| LRPPI2 | LRPPI3 | -0.6745 | 0.7892 | 2.2529 | FALSE |
| LRPPI2 | LRPPI4 | -0.7284 | 0.7353 | 2.1990 | FALSE |
| LRPPI2 | ME1 | -0.7529 | 0.7108 | 2.1745 | FALSE |
| LRPPI2 | ME2 | 0.0608 | 1.5245 | 2.9882 | TRUE |
| LRPPI2 | ME3 | 3.1638 | 4.6275 | 6.0911 | TRUE |
| LRPPI3 | LRPPI4 | -1.5176 | -0.0539 | 1.4098 | FALSE |
| LRPPI3 | ME1 | -1.5421 | -0.0784 | 1.3852 | FALSE |
| LRPPI3 | ME2 | -0.7284 | 0.7353 | 2.1990 | FALSE |
| LRPPI3 | ME3 | 2.3746 | 3.8382 | 5.3019 | TRUE |
| LRPPI4 | ME1 | -1.4882 | -0.0245 | 1.4392 | FALSE |
| LRPPI4 | ME2 | -0.6745 | 0.7892 | 2.2529 | FALSE |
| LRPPI4 | ME3 | 2.4285 | 3.8922 | 5.3558 | TRUE |
| ME1 | ME2 | -0.6499 | 0.8137 | 2.2774 | FALSE |
| ME1 | ME3 | 2.4530 | 3.9167 | 5.3803 | TRUE |
| ME2 | ME3 | 1.6393 | 3.1029 | 4.5666 | TRUE |
File:2008 Artist.perClassAccuracy.friedman.tukeyKramerHSD.png
Folds vs. Systems
The Friedman test was run in MATLAB against the accuracy for each fold.
Friedman's Anova Table
| Source | SS | df | MS | Chi-sq | Prob>Chi-sq |
|---|---|---|---|---|---|
| Columns | 319.333 | 10 | 31.9333 | 29.03 | 0.0012 |
| Error | 10.667 | 20 | 0.5333 | ||
| Total | 330 | 32 |
Tukey-Kramer HSD Multi-Comparison
The Tukey-Kramer HSD multi-comparison data below was generated using the following MATLAB instruction. Command: [c, m, h, gnames] = multicompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);
| TeamID | TeamID | Lowerbound | Mean | Upperbound | Significance |
|---|---|---|---|---|---|
| GP2 | GT1 | -7.3828 | 1.3333 | 10.0495 | FALSE |
| GP2 | GT2 | -7.0495 | 1.6667 | 10.3828 | FALSE |
| GP2 | GT3 | -5.7162 | 3.0000 | 11.7162 | FALSE |
| GP2 | LRPPI1 | -4.0495 | 4.6667 | 13.3828 | FALSE |
| GP2 | LRPPI2 | -4.0495 | 4.6667 | 13.3828 | FALSE |
| GP2 | LRPPI3 | -2.0495 | 6.6667 | 15.3828 | FALSE |
| GP2 | LRPPI4 | -2.3828 | 6.3333 | 15.0495 | FALSE |
| GP2 | ME1 | -1.0495 | 7.6667 | 16.3828 | FALSE |
| GP2 | ME2 | 0.2838 | 9.0000 | 17.7162 | TRUE |
| GP2 | ME3 | 1.2838 | 10.0000 | 18.7162 | TRUE |
| GT1 | GT2 | -8.3828 | 0.3333 | 9.0495 | FALSE |
| GT1 | GT3 | -7.0495 | 1.6667 | 10.3828 | FALSE |
| GT1 | LRPPI1 | -5.3828 | 3.3333 | 12.0495 | FALSE |
| GT1 | LRPPI2 | -5.3828 | 3.3333 | 12.0495 | FALSE |
| GT1 | LRPPI3 | -3.3828 | 5.3333 | 14.0495 | FALSE |
| GT1 | LRPPI4 | -3.7162 | 5.0000 | 13.7162 | FALSE |
| GT1 | ME1 | -2.3828 | 6.3333 | 15.0495 | FALSE |
| GT1 | ME2 | -1.0495 | 7.6667 | 16.3828 | FALSE |
| GT1 | ME3 | -0.0495 | 8.6667 | 17.3828 | FALSE |
| GT2 | GT3 | -7.3828 | 1.3333 | 10.0495 | FALSE |
| GT2 | LRPPI1 | -5.7162 | 3.0000 | 11.7162 | FALSE |
| GT2 | LRPPI2 | -5.7162 | 3.0000 | 11.7162 | FALSE |
| GT2 | LRPPI3 | -3.7162 | 5.0000 | 13.7162 | FALSE |
| GT2 | LRPPI4 | -4.0495 | 4.6667 | 13.3828 | FALSE |
| GT2 | ME1 | -2.7162 | 6.0000 | 14.7162 | FALSE |
| GT2 | ME2 | -1.3828 | 7.3333 | 16.0495 | FALSE |
| GT2 | ME3 | -0.3828 | 8.3333 | 17.0495 | FALSE |
| GT3 | LRPPI1 | -7.0495 | 1.6667 | 10.3828 | FALSE |
| GT3 | LRPPI2 | -7.0495 | 1.6667 | 10.3828 | FALSE |
| GT3 | LRPPI3 | -5.0495 | 3.6667 | 12.3828 | FALSE |
| GT3 | LRPPI4 | -5.3828 | 3.3333 | 12.0495 | FALSE |
| GT3 | ME1 | -4.0495 | 4.6667 | 13.3828 | FALSE |
| GT3 | ME2 | -2.7162 | 6.0000 | 14.7162 | FALSE |
| GT3 | ME3 | -1.7162 | 7.0000 | 15.7162 | FALSE |
| LRPPI1 | LRPPI2 | -8.7162 | 0.0000 | 8.7162 | FALSE |
| LRPPI1 | LRPPI3 | -6.7162 | 2.0000 | 10.7162 | FALSE |
| LRPPI1 | LRPPI4 | -7.0495 | 1.6667 | 10.3828 | FALSE |
| LRPPI1 | ME1 | -5.7162 | 3.0000 | 11.7162 | FALSE |
| LRPPI1 | ME2 | -4.3828 | 4.3333 | 13.0495 | FALSE |
| LRPPI1 | ME3 | -3.3828 | 5.3333 | 14.0495 | FALSE |
| LRPPI2 | LRPPI3 | -6.7162 | 2.0000 | 10.7162 | FALSE |
| LRPPI2 | LRPPI4 | -7.0495 | 1.6667 | 10.3828 | FALSE |
| LRPPI2 | ME1 | -5.7162 | 3.0000 | 11.7162 | FALSE |
| LRPPI2 | ME2 | -4.3828 | 4.3333 | 13.0495 | FALSE |
| LRPPI2 | ME3 | -3.3828 | 5.3333 | 14.0495 | FALSE |
| LRPPI3 | LRPPI4 | -9.0495 | -0.3333 | 8.3828 | FALSE |
| LRPPI3 | ME1 | -7.7162 | 1.0000 | 9.7162 | FALSE |
| LRPPI3 | ME2 | -6.3828 | 2.3333 | 11.0495 | FALSE |
| LRPPI3 | ME3 | -5.3828 | 3.3333 | 12.0495 | FALSE |
| LRPPI4 | ME1 | -7.3828 | 1.3333 | 10.0495 | FALSE |
| LRPPI4 | ME2 | -6.0495 | 2.6667 | 11.3828 | FALSE |
| LRPPI4 | ME3 | -5.0495 | 3.6667 | 12.3828 | FALSE |
| ME1 | ME2 | -7.3828 | 1.3333 | 10.0495 | FALSE |
| ME1 | ME3 | -6.3828 | 2.3333 | 11.0495 | FALSE |
| ME2 | ME3 | -7.7162 | 1.0000 | 9.7162 | FALSE |
File:2008 Artist.perFoldAccuracy.friedman.tukeyKramerHSD.png