2005:Symbolic Genre Classification Results
From MIREX Wiki
Introduction
Goal
To classify MIDI recordings into genre categories.
Dataset
Two sets of genre categories were used, one consisting of 38 categories and one consisting of 9 categories. Each category was represented by 25 MIDI files.Thus, the 38 genre test contained 950 MIDI files and the 9 genre test contained 225 MIDI files.Test runs were 3-fold cross validated with each algorithm tested using identical training and testing data splits.
Results
Overall
| OVERALL | |||
|---|---|---|---|
| Rank | Participant | Mean Hierarchical Classification Accuracy | Mean Raw Classification Accuracy |
| 1 | McKay & Fujinaga | 77.17% | 65.28% |
| 2 | Basili, Serafini, & Stellato (NB) | 72.08% | 58.53% |
| 3 | Li, M. | 67.57% | 55.90% |
| 4 | Basili, Serafini, & Stellato (J48) | 67.14% | 53.14% |
| 5 | Ponce de Leon & Inesta | 37.76% | 26.52% |
38 Classes
| 38 Classes | ||||||||
|---|---|---|---|---|---|---|---|---|
| Rank | Participant | Hierarchical Classification Accuracy | Hierarchical Classification Accuracy Std | Raw Classification Accuracy | Raw Classification Accuracy Std | Runtime (s) | Machine | Confusion Matrix Files |
| 1 | McKay & Fujinaga | 64.33% | 1.04 | 46.11% | 1.51 | 3 days | R | MF_38eval.txt |
| 2 | Basili, Serafini, & Stellato (NB) | 62.60% | 0.26 | 45.05% | 0.55 | N/A | N/A | BST_NB_38eval.txt |
| 3 | Basili, Serafini, & Stellato (J48) | 57.61% | 1.14 | 40.95% | 1.35 | N/A | N/A | BST_J48_38eval.txt |
| 4 | Li, M. | 54.91% | 0.66 | 39.79% | 0.87 | 15,948 | G | L_38eval.txt |
| 5 | Ponce de Leon & Inesta | 24.84% | 1.40 | 15.26% | 1.13 | 821 | L | PI_38eval.txt |
9 Classes
| 9 Classes | ||||||||
|---|---|---|---|---|---|---|---|---|
| Rank | Participant | Hierarchical Classification Accuracy | Hierarchical Classification Accuracy Std | Raw Classification Accuracy | Raw Classification Accuracy Std | Runtime (s) | Machine | Confusion Matrix Files |
| 1 | McKay & Fujinaga | 90.00% | 0.60 | 84.44% | 1.41 | 18,375 | R | MF_9eval.txt |
| 2 | Basili, Serafini, & Stellato (NB) | 81.56% | 0.76 | 72.00% | 0.88 | N/A | N/A | BST_NB_9eval.txt |
| 3 | Li, M. | 80.22% | 1.47 | 72.00% | 2.31 | 3,777 | G | L_9eval.txt |
| 4 | Basili, Serafini, & Stellato (J48) | 76.67% | 1.11 | 65.33% | 1.65 | N/A | N/A | BST_J48_9eval.txt |
| 5 | Ponce de Leon & Inesta | 50.67% | 1.26 | 37.78% | 2.30 | 197 | L | PI_9eval.txt |
