Difference between revisions of "2005:Audio Drum Detection Results"
From MIREX Wiki
(→Overall Collections) |
|||
| Line 1: | Line 1: | ||
==Introduction== | ==Introduction== | ||
| − | + | ===Goal=== | |
| − | + | To detect the occurrences of drum events in polyphonic audio. | |
| − | + | ||
| + | ===Dataset=== | ||
| + | At least 50 files of both live and sequenced music, with many genres encompassed and various degrees of drum audio contained in the files. Three collections of music were used: Christian Dittmar (CD), Koen Tanghe (KT) and Masataka Goto (MG). Participants were evaluated against music from each individual collection, and then the three collection scores are averaged to produce a composite score. | ||
==Results== | ==Results== | ||
Latest revision as of 10:26, 2 August 2010
Contents
Introduction
Goal
To detect the occurrences of drum events in polyphonic audio.
Dataset
At least 50 files of both live and sequenced music, with many genres encompassed and various degrees of drum audio contained in the files. Three collections of music were used: Christian Dittmar (CD), Koen Tanghe (KT) and Masataka Goto (MG). Participants were evaluated against music from each individual collection, and then the three collection scores are averaged to produce a composite score.
Results
Overall Collections
| OVERALL | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Rank | Participant | Total Average Classification F-measure | Total Overall Onset Precision | Total Overall Onset Recall | Total Overall Onset F-measure | BD Average F-measure | HH Average F-measure | SD Average F-measure | Runtime (s) | Machine |
| 1 | Yoshii, Goto, & Okuno | 0.670 | 64.92% | 67.02% | 0.659 | 0.728 | 0.574 | 0.702 | 8534 | B 0 |
| 2 | Tanghe, Degroeve, & De Baets 3 | 0.611 | 63.30% | 71.19% | 0.670 | 0.688 | 0.601 | 0.555 | 1337 | Y |
| 3 | Tanghe, Degroeve, & De Baets 4 | 0.609 | 62.57% | 71.09% | 0.666 | 0.686 | 0.590 | 0.562 | 1342 | Y |
| 4 | Tanghe, Degroeve, & De Baets 1 | 0.599 | 60.02% | 72.45% | 0.657 | 0.677 | 0.588 | 0.542 | 1350 | Y |
| 5 | Dittmar, C. | 0.588 | 65.68% | 63.38% | 0.645 | 0.606 | 0.585 | 0.581 | 673 | R |
| 6 | Paulus, J. | 0.499 | 59.61% | 64.86% | 0.621 | 0.527 | 0.587 | 0.430 | 1137 | L |
| 7 | Gillet & Richard 2 | 0.443 | 77.09% | 40.63% | 0.532 | 0.598 | 0.334 | 0.428 | 21248 | F |
| 8 | Gillet & Richard 1 | 0.391 | 69.84% | 37.98% | 0.492 | 0.533 | 0.343 | 0.317 | 21997 | F |
Christan Dittmar Collection
| CHRISTIAN DITTMAR COLLECTION | ||||||||
|---|---|---|---|---|---|---|---|---|
| Rank | Participant | Total Average Classification F-measure | Total Overall Onset Precision | Total Overall Onset Recall | Total Overall Onset F-measure | BD Average F-measure | HH Average F-measure | SD Average F-measure |
| 1 | Dittmar, C. | 0.753 | 77.73% | 72.56% | 0.751 | 0.783 | 0.696 | 0.790 |
| 2 | Yoshii, Goto, & Okuno | 0.690 | 64.25% | 62.75% | 0.660 | 0.714 | 0.533 | 0.811 |
| 3 | Tanghe, Degroeve, & De Baets 3 | 0.595 | 61.85% | 64.85% | 0.633 | 0.685 | 0.568 | 0.548 |
| 4 | Tanghe, Degroeve, & De Baets 4 | 0.589 | 62.45% | 64.22% | 0.628 | 0.668 | 0.555 | 0.559 |
| 5 | Tanghe, Degroeve, & De Baets 1 | 0.580 | 57.78% | 65.94% | 0.616 | 0.669 | 0.553 | 0.533 |
| 6 | Paulus, J. | 0.440 | 55.82% | 54.36% | 0.551 | 0.430 | 0.497 | 0.424 |
| 7 | Gillet & Richard 2 | 0.401 | 77.22% | 30.16% | 0.434 | 0.658 | 0.156 | 0.364 |
| 8 | Gillet & Richard 1 | 0.339 | 66.33% | 30.57% | 0.418 | 0.466 | 0.279 | 0.265 |
Koen Tanghe Collection
| KOEN TANGHE COLLECTION | ||||||||
|---|---|---|---|---|---|---|---|---|
| Rank | Participant | Total Average Classification F-measure | Total Overall Onset Precision | Total Overall Onset Recall | Total Overall Onset F-measure | BD Average F-measure | HH Average F-measure | SD Average F-measure |
| 1 | Yoshii, Goto, & Okuno | 0.617 | 53.06% | 66.30% | 0.589 | 0.686 | 0.481 | 0.652 |
| 2 | Tanghe, Degroeve, & De Baets 3 | 0.546 | 52.67% | 68.21% | 0.594 | 0.613 | 0.485 | 0.525 |
| 3 | Tanghe, Degroeve, & De Baets 4 | 0.541 | 51.90% | 67.98% | 0.589 | 0.612 | 0.479 | 0.523 |
| 4 | Tanghe, Degroeve, & De Baets 1 | 0.533 | 47.48% | 69.24% | 0.577 | 0.602 | 0.467 | 0.511 |
| 5 | Dittmar, C. | 0.494 | 51.20% | 51.51% | 0.514 | 0.509 | 0.418 | 0.535 |
| 6 | Paulus, J. | 0.425 | 55.82% | 55.10% | 0.555 | 0.444 | 0.489 | 0.412 |
| 7 | Gillet & Richard 2 | 0.273 | 66.68% | 28.22% | 0.397 | 0.393 | 0.178 | 0.269 |
| 8 | Gillet & Richard 1 | 0.259 | 58.48% | 26.11% | 0.361 | 0.375 | 0.196 | 0.210 |
Masataka Goto Collection
| MASATAKA GOTO COLLECTION (50 songs from RWC Music Database: RWC-MDB-P-2001) | ||||||||
|---|---|---|---|---|---|---|---|---|
| Rank | Participant | Total Average Classification F-measure | Total Overall Onset Precision | Total Overall Onset Recall | Total Overall Onset F-measure | BD Average F-measure | HH Average F-measure | SD Average F-measure |
| 1 | Yoshii, Goto, & Okuno | 0.716 | 76.13% | 69.16% | 0.725 | 0.776 | 0.661 | 0.710 |
| 2 | Tanghe, Degroeve, & De Baets 4 | 0.685 | 72.32% | 75.83% | 0.740 | 0.766 | 0.691 | 0.599 |
| 3 | Tanghe, Degroeve, & De Baets 3 | 0.683 | 72.74% | 75.65% | 0.742 | 0.763 | 0.701 | 0.585 |
| 4 | Tanghe, Degroeve, & De Baets 1 | 0.673 | 69.86% | 77.12% | 0.733 | 0.753 | 0.693 | 0.574 |
| 5 | Gillet & Richard 2 | 0.630 | 81.60% | 53.00% | 0.643 | 0.774 | 0.517 | 0.599 |
| 6 | Dittmar, C. | 0.617 | 71.37% | 67.78% | 0.695 | 0.631 | 0.675 | 0.544 |
| 7 | Paulus, J. | 0.597 | 62.90% | 75.47% | 0.686 | 0.648 | 0.695 | 0.449 |
| 8 | Gillet & Richard 1 | 0.544 | 76.11% | 48.80% | 0.595 | 0.715 | 0.479 | 0.436 |
