Difference between revisions of "2005:Symbolic Genre Classification Results"
From MIREX Wiki
(Created page with '{| border="1" |- style="background: yellow; text-align: center;" ! colspan="4" | OVERALL |-style="background: yellow;" !Rank !Participant !Mean Hierarchical Classification Accu…') |
|||
| Line 1: | Line 1: | ||
| + | ==Introduction== | ||
| + | ''' Goal:''' To classify MIDI recordings into genre categories. | ||
| + | |||
| + | '''Dataset:''' Two sets of genre categories were used, one consisting of 38 categories and one consisting of 9 categories. Each category was represented by 25 MIDI files.Thus, the 38 genre test contained 950 MIDI files and the 9 genre test contained 225 MIDI files.Test runs were 3-fold crossvalidated with each algorithm tested using identical training and testing data splits. | ||
| + | |||
{| border="1" | {| border="1" | ||
|- style="background: yellow; text-align: center;" | |- style="background: yellow; text-align: center;" | ||
| Line 48: | Line 53: | ||
!Machine | !Machine | ||
!Confusion Matrix Files | !Confusion Matrix Files | ||
| − | |||
| − | |||
|---- | |---- | ||
|1 | |1 | ||
| Line 60: | Line 63: | ||
|R | |R | ||
|MF_38eval.txt | |MF_38eval.txt | ||
| − | |||
| − | |||
|---- | |---- | ||
|2 | |2 | ||
| Line 72: | Line 73: | ||
|N/A | |N/A | ||
|BST_NB_38eval.txt | |BST_NB_38eval.txt | ||
| − | |||
| − | |||
|---- | |---- | ||
|3 | |3 | ||
| Line 84: | Line 83: | ||
|N/A | |N/A | ||
|BST_J48_38eval.txt | |BST_J48_38eval.txt | ||
| − | |||
| − | |||
|---- | |---- | ||
|4 | |4 | ||
| Line 96: | Line 93: | ||
|G | |G | ||
|L_38eval.txt | |L_38eval.txt | ||
| − | |||
| − | |||
|---- | |---- | ||
|5 | |5 | ||
| Line 108: | Line 103: | ||
|L | |L | ||
|PI_38eval.txt | |PI_38eval.txt | ||
| − | |||
| − | |||
|---- | |---- | ||
| − | | | + | |} |
| − | + | <br> | |
| − | + | {| border="1" | |
| − | + | |- style="background: yellow; text-align: center;" | |
| − | + | ! colspan="9" | 9 Classes | |
| − | + | |-style="background: yellow;" | |
| − | + | !Rank | |
| − | + | !Participant | |
| − | | | + | !Hierarchical Classification Accuracy |
| − | | | + | !Hierarchical Classification Accuracy Std |
| − | | | + | !Raw Classification Accuracy |
| − | |- | + | !Raw Classification Accuracy Std |
| − | + | !Runtime (s) | |
| − | + | !Machine | |
| − | + | !Confusion Matrix Files | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
|---- | |---- | ||
|1 | |1 | ||
| Line 144: | Line 129: | ||
|R | |R | ||
|MF_9eval.txt | |MF_9eval.txt | ||
| − | |||
| − | |||
|---- | |---- | ||
|2 | |2 | ||
| Line 156: | Line 139: | ||
|N/A | |N/A | ||
|BST_NB_9eval.txt | |BST_NB_9eval.txt | ||
| − | |||
| − | |||
|---- | |---- | ||
|3 | |3 | ||
| Line 168: | Line 149: | ||
|G | |G | ||
|L_9eval.txt | |L_9eval.txt | ||
| − | |||
| − | |||
|---- | |---- | ||
|4 | |4 | ||
| Line 180: | Line 159: | ||
|N/A | |N/A | ||
|BST_J48_9eval.txt | |BST_J48_9eval.txt | ||
| − | |||
| − | |||
|---- | |---- | ||
|5 | |5 | ||
| Line 192: | Line 169: | ||
|L | |L | ||
|PI_9eval.txt | |PI_9eval.txt | ||
| − | |||
| − | |||
|---- | |---- | ||
|} | |} | ||
Revision as of 18:37, 29 July 2010
Introduction
Goal: To classify MIDI recordings into genre categories.
Dataset: Two sets of genre categories were used, one consisting of 38 categories and one consisting of 9 categories. Each category was represented by 25 MIDI files.Thus, the 38 genre test contained 950 MIDI files and the 9 genre test contained 225 MIDI files.Test runs were 3-fold crossvalidated with each algorithm tested using identical training and testing data splits.
| OVERALL | |||
|---|---|---|---|
| Rank | Participant | Mean Hierarchical Classification Accuracy | Mean Raw Classification Accuracy |
| 1 | McKay & Fujinaga | 77.17% | 65.28% |
| 2 | Basili, Serafini, & Stellato (NB) | 72.08% | 58.53% |
| 3 | Li, M. | 67.57% | 55.90% |
| 4 | Basili, Serafini, & Stellato (J48) | 67.14% | 53.14% |
| 5 | Ponce de Leon & Inesta | 37.76% | 26.52% |
| 38 Classes | ||||||||
|---|---|---|---|---|---|---|---|---|
| Rank | Participant | Hierarchical Classification Accuracy | Hierarchical Classification Accuracy Std | Raw Classification Accuracy | Raw Classification Accuracy Std | Runtime (s) | Machine | Confusion Matrix Files |
| 1 | McKay & Fujinaga | 64.33% | 1.04 | 46.11% | 1.51 | 3 days | R | MF_38eval.txt |
| 2 | Basili, Serafini, & Stellato (NB) | 62.60% | 0.26 | 45.05% | 0.55 | N/A | N/A | BST_NB_38eval.txt |
| 3 | Basili, Serafini, & Stellato (J48) | 57.61% | 1.14 | 40.95% | 1.35 | N/A | N/A | BST_J48_38eval.txt |
| 4 | Li, M. | 54.91% | 0.66 | 39.79% | 0.87 | 15,948 | G | L_38eval.txt |
| 5 | Ponce de Leon & Inesta | 24.84% | 1.40 | 15.26% | 1.13 | 821 | L | PI_38eval.txt |
| 9 Classes | ||||||||
|---|---|---|---|---|---|---|---|---|
| Rank | Participant | Hierarchical Classification Accuracy | Hierarchical Classification Accuracy Std | Raw Classification Accuracy | Raw Classification Accuracy Std | Runtime (s) | Machine | Confusion Matrix Files |
| 1 | McKay & Fujinaga | 90.00% | 0.60 | 84.44% | 1.41 | 18,375 | R | MF_9eval.txt |
| 2 | Basili, Serafini, & Stellato (NB) | 81.56% | 0.76 | 72.00% | 0.88 | N/A | N/A | BST_NB_9eval.txt |
| 3 | Li, M. | 80.22% | 1.47 | 72.00% | 2.31 | 3,777 | G | L_9eval.txt |
| 4 | Basili, Serafini, & Stellato (J48) | 76.67% | 1.11 | 65.33% | 1.65 | N/A | N/A | BST_J48_9eval.txt |
| 5 | Ponce de Leon & Inesta | 50.67% | 1.26 | 37.78% | 2.30 | 197 | L | PI_9eval.txt |
