2005:Audio Drum Detection Results

From MIREX Wiki

Introduction

Goal

To detect the occurrences of drum events in polyphonic audio.

Dataset

At least 50 files of both live and sequenced music, with many genres encompassed and various degrees of drum audio contained in the files. Three collections of music were used: Christian Dittmar (CD), Koen Tanghe (KT) and Masataka Goto (MG). Participants were evaluated against music from each individual collection, and then the three collection scores are averaged to produce a composite score.

Results

Overall Collections

OVERALL
Rank Participant Total Average Classification F-measure Total Overall Onset Precision Total Overall Onset Recall Total Overall Onset F-measure BD Average F-measure HH Average F-measure SD Average F-measure Runtime (s) Machine
1 Yoshii, Goto, & Okuno 0.670 64.92% 67.02% 0.659 0.728 0.574 0.702 8534 B 0
2 Tanghe, Degroeve, & De Baets 3 0.611 63.30% 71.19% 0.670 0.688 0.601 0.555 1337 Y
3 Tanghe, Degroeve, & De Baets 4 0.609 62.57% 71.09% 0.666 0.686 0.590 0.562 1342 Y
4 Tanghe, Degroeve, & De Baets 1 0.599 60.02% 72.45% 0.657 0.677 0.588 0.542 1350 Y
5 Dittmar, C. 0.588 65.68% 63.38% 0.645 0.606 0.585 0.581 673 R
6 Paulus, J. 0.499 59.61% 64.86% 0.621 0.527 0.587 0.430 1137 L
7 Gillet & Richard 2 0.443 77.09% 40.63% 0.532 0.598 0.334 0.428 21248 F
8 Gillet & Richard 1 0.391 69.84% 37.98% 0.492 0.533 0.343 0.317 21997 F

Christan Dittmar Collection

CHRISTIAN DITTMAR COLLECTION
Rank Participant Total Average Classification F-measure Total Overall Onset Precision Total Overall Onset Recall Total Overall Onset F-measure BD Average F-measure HH Average F-measure SD Average F-measure
1 Dittmar, C. 0.753 77.73% 72.56% 0.751 0.783 0.696 0.790
2 Yoshii, Goto, & Okuno 0.690 64.25% 62.75% 0.660 0.714 0.533 0.811
3 Tanghe, Degroeve, & De Baets 3 0.595 61.85% 64.85% 0.633 0.685 0.568 0.548
4 Tanghe, Degroeve, & De Baets 4 0.589 62.45% 64.22% 0.628 0.668 0.555 0.559
5 Tanghe, Degroeve, & De Baets 1 0.580 57.78% 65.94% 0.616 0.669 0.553 0.533
6 Paulus, J. 0.440 55.82% 54.36% 0.551 0.430 0.497 0.424
7 Gillet & Richard 2 0.401 77.22% 30.16% 0.434 0.658 0.156 0.364
8 Gillet & Richard 1 0.339 66.33% 30.57% 0.418 0.466 0.279 0.265

Koen Tanghe Collection

KOEN TANGHE COLLECTION
Rank Participant Total Average Classification F-measure Total Overall Onset Precision Total Overall Onset Recall Total Overall Onset F-measure BD Average F-measure HH Average F-measure SD Average F-measure
1 Yoshii, Goto, & Okuno 0.617 53.06% 66.30% 0.589 0.686 0.481 0.652
2 Tanghe, Degroeve, & De Baets 3 0.546 52.67% 68.21% 0.594 0.613 0.485 0.525
3 Tanghe, Degroeve, & De Baets 4 0.541 51.90% 67.98% 0.589 0.612 0.479 0.523
4 Tanghe, Degroeve, & De Baets 1 0.533 47.48% 69.24% 0.577 0.602 0.467 0.511
5 Dittmar, C. 0.494 51.20% 51.51% 0.514 0.509 0.418 0.535
6 Paulus, J. 0.425 55.82% 55.10% 0.555 0.444 0.489 0.412
7 Gillet & Richard 2 0.273 66.68% 28.22% 0.397 0.393 0.178 0.269
8 Gillet & Richard 1 0.259 58.48% 26.11% 0.361 0.375 0.196 0.210

Masataka Goto Collection

MASATAKA GOTO COLLECTION (50 songs from RWC Music Database: RWC-MDB-P-2001)
Rank Participant Total Average Classification F-measure Total Overall Onset Precision Total Overall Onset Recall Total Overall Onset F-measure BD Average F-measure HH Average F-measure SD Average F-measure
1 Yoshii, Goto, & Okuno 0.716 76.13% 69.16% 0.725 0.776 0.661 0.710
2 Tanghe, Degroeve, & De Baets 4 0.685 72.32% 75.83% 0.740 0.766 0.691 0.599
3 Tanghe, Degroeve, & De Baets 3 0.683 72.74% 75.65% 0.742 0.763 0.701 0.585
4 Tanghe, Degroeve, & De Baets 1 0.673 69.86% 77.12% 0.733 0.753 0.693 0.574
5 Gillet & Richard 2 0.630 81.60% 53.00% 0.643 0.774 0.517 0.599
6 Dittmar, C. 0.617 71.37% 67.78% 0.695 0.631 0.675 0.544
7 Paulus, J. 0.597 62.90% 75.47% 0.686 0.648 0.695 0.449
8 Gillet & Richard 1 0.544 76.11% 48.80% 0.595 0.715 0.479 0.436