Difference between revisions of "2005:Audio Artist Identification Results"
From MIREX Wiki
| Line 1: | Line 1: | ||
==Introduction== | ==Introduction== | ||
| − | ==Goal== | + | ===Goal=== |
'''Goal:''' To identify artist from music audio (in PCM format). | '''Goal:''' To identify artist from music audio (in PCM format). | ||
| − | == | + | ===Datasets=== |
'''Dataset:''' Two sets of data were used: Magnatune and USPOP. The audio sampling rates used were either 44.1 KHz or 22.05 KHz (mono). More data information is in the following table. | '''Dataset:''' Two sets of data were used: Magnatune and USPOP. The audio sampling rates used were either 44.1 KHz or 22.05 KHz (mono). More data information is in the following table. | ||
| Line 18: | Line 18: | ||
|} | |} | ||
| − | == | + | ==Results== |
| − | ==Overall== | + | ===Overall=== |
{| border="1" cellspacing="0" | {| border="1" cellspacing="0" | ||
|- style="background: yellow; text-align: center;" | |- style="background: yellow; text-align: center;" | ||
| Line 44: | Line 44: | ||
| − | ==Magnatune Dataset== | + | ===Magnatune Dataset=== |
{| border="1" cellspacing="0" | {| border="1" cellspacing="0" | ||
|- style="background: yellow; text-align: center;" | |- style="background: yellow; text-align: center;" | ||
| Line 73: | Line 73: | ||
|} | |} | ||
| − | ==USPOP Dataset== | + | ===USPOP Dataset=== |
{| border="1" cellspacing="0" | {| border="1" cellspacing="0" | ||
|- style="background: yellow; text-align: center;" | |- style="background: yellow; text-align: center;" | ||
Revision as of 07:27, 30 July 2010
Contents
Introduction
Goal
Goal: To identify artist from music audio (in PCM format).
Datasets
Dataset: Two sets of data were used: Magnatune and USPOP. The audio sampling rates used were either 44.1 KHz or 22.05 KHz (mono). More data information is in the following table.
| Dataset | Size (@ 44.1 KHz) | Number of Training Files | Number of Testing Files |
|---|---|---|---|
| Magnatune | 35.2 GB | 1158 | 642 |
| USPOP | 37.3 GB | 1158 | 653 |
Results
Overall
| OVERALL | ||
|---|---|---|
| Rank | Participant | Mean of Magnatune Raw Classification Accuracy and USPOP Raw Classification Accuracy |
| 1 | Mandel & Ellis | 72.45% |
| 2 | Bergstra, Casagrande, & Eck (1) | 68.57% |
| 3 | Bergstra, Casagrande, & Eck (2) | 66.71% |
| 4 | Pampalk, E. | 61.28% |
| 5 | West & Lamere | 47.24% |
| 6 | Tzanetakis, G. | 42.05% |
| 7 | Logan, B | 25.95% |
Magnatune Dataset
| Magnatune Dataset | ||||||
|---|---|---|---|---|---|---|
| Rank | Participant | Raw Classification Accuracy | Normalized Raw classification Accuracy | Runtime (s) | Machine | Confusion Matrix Files |
| 1 | Bergstra, Casagrande, & Eck (1) | 77.26% | 79.64% | 24 hours | B0 | BCE_1_MTeval.txt |
| 2 | Mandel & Ellis | 76.60% | 76.62% | 11073 | R | ME_MTeval.txt |
| 3 | Bergstra, Casagrande, & Eck (2) | 74.45% | 74.51% | -- | -- | BCE_2_MTeval.txt |
| 4 | Pampalk, E. | 66.36% | 66.48% | 4272 | B1 | P_MTeval.txt |
| 5 | Tzanetakis, G. | 55.45% | 55.59% | 2632 | B0 | T_MTeval.txt |
| 6 | West & Lamere | 53.43% | 53.48% | 27480 | B3 | WL_MTeval.txt |
| 7 | Logan, B | 37.07% | 37.10% | N/A | B3 | L_MTeval.txt |
| 8 | Lidy & Rauber (SSD+RH) | TO * | -- | -- | -- | -- |
| 8 | Lidy & Rauber (RP+SSD) | TO * | -- | -- | -- | -- |
| 8 | Lidy & Rauber (RP+SSD+RH) | TO * | -- | -- | -- | -- |
USPOP Dataset
| USPOP Dataset | ||||||
|---|---|---|---|---|---|---|
| Rank | Participant | Raw Classification Accuracy | Normalized Raw classification Accuracy | Runtime (s) | Machine | Confusion Matrix Files |
| 1 | Mandel & Ellis | 68.30% | 67.96% | 10240 | R | ME_USeval.txt |
| 2 | Bergstra, Casagrande, & Eck (1) | 59.88% | 60.90% | 24 Hours | B0 | ME_USeval.txt |
| 3 | Bergstra, Casagrande, & Eck (2) | 58.96% | 58.96% | -- | -- | BCE_2_USeval.txt |
| 4 | Pampalk, E. | 56.20% | 56.03% | 4321 | B1 | P_USeval.txt |
| 5 | West & Lamere | 41.04% | 41.00% | 26871 | B3 | WL_USeval.txt |
| 6 | Tzanetakis, G. | 28.64% | 28.48% | 2443 | B0 | T_USeval.txt |
| 7 | Logan, B. | 14.83% | 14.76% | N/A | B3 | L_USeval.txt |
| 8 | Lidy & Rauber (SSD+RH) | TO * | -- | -- | -- | -- |
| 8 | Lidy & Rauber (RP+SSD) | TO * | -- | -- | -- | -- |
| 8 | Lidy & Rauber (RP+SSD+RH) | TO * | -- | -- | -- | -- |
Note:
DNC: did not complete ( error in execution).
TO: timed out (did not complete within 24 hours).
