2005:Audio Drum Detection Results

Goal: To detect the occurences of drum events in polyphonic audio.

Dataset: At least 50 files of both live and sequenced music, with many genres encompassed and various degrees of drum audio contained in the files. Three collections of music were used: Christian Dittmar (CD), Koen Tanghe (KT) and Masataka Goto (MG). Participants were evaluated against music from each individual collection, and then the three collection scores are averaged to produce a composite score.

OVERALL
Rank	Participant	Total Average Classification F-measure	Total Overall Onset Precision	Total Overall Onset Recall	Total Overall Onset F-measure	BD Average F-measure	HH Average F-measure	SD Average F-measure	Runtime (s)	Machine
1	Yoshii, Goto, & Okuno	0.670	64.92%	67.02%	0.659	0.728	0.574	0.702	8534	B 0
2	Tanghe, Degroeve, & De Baets 3	0.611	63.30%	71.19%	0.670	0.688	0.601	0.555	1337	Y
3	Tanghe, Degroeve, & De Baets 4	0.609	62.57%	71.09%	0.666	0.686	0.590	0.562	1342	Y
4	Tanghe, Degroeve, & De Baets 1	0.599	60.02%	72.45%	0.657	0.677	0.588	0.542	1350	Y
5	Dittmar, C.	0.588	65.68%	63.38%	0.645	0.606	0.585	0.581	673	R
6	Paulus, J.	0.499	59.61%	64.86%	0.621	0.527	0.587	0.430	1137	L
7	Gillet & Richard 2	0.443	77.09%	40.63%	0.532	0.598	0.334	0.428	21248	F
8	Gillet & Richard 1	0.391	69.84%	37.98%	0.492	0.533	0.343	0.317	21997	F

CHRISTIAN DITTMAR COLLECTION
Rank	Participant	Total Average Classification F-measure	Total Overall Onset Precision	Total Overall Onset Recall	Total Overall Onset F-measure	BD Average F-measure	HH Average F-measure	SD Average F-measure
1	Dittmar, C.	0.753	77.73%	72.56%	0.751	0.783	0.696	0.790
2	Yoshii, Goto, & Okuno	0.690	64.25%	62.75%	0.660	0.714	0.533	0.811
3	Tanghe, Degroeve, & De Baets 3	0.595	61.85%	64.85%	0.633	0.685	0.568	0.548
4	Tanghe, Degroeve, & De Baets 4	0.589	62.45%	64.22%	0.628	0.668	0.555	0.559
5	Tanghe, Degroeve, & De Baets 1	0.580	57.78%	65.94%	0.616	0.669	0.553	0.533
6	Paulus, J.	0.440	55.82%	54.36%	0.551	0.430	0.497	0.424
7	Gillet & Richard 2	0.401	77.22%	30.16%	0.434	0.658	0.156	0.364
8	Gillet & Richard 1	0.339	66.33%	30.57%	0.418	0.466	0.279	0.265

1 Yoshii, Goto, & Okuno 0.617 53.06% 66.30% 0.589 0.686 0.481 0.652 2 Tanghe, Degroeve, & De Baets 3 0.546 52.67% 68.21% 0.594 0.613 0.485 0.525 3 Tanghe, Degroeve, & De Baets 4 0.541 51.90% 67.98% 0.589 0.612 0.479 0.523 4 Tanghe, Degroeve, & De Baets 1 0.533 47.48% 69.24% 0.577 0.602 0.467 0.511 5 Dittmar, C. 0.494 51.20% 51.51% 0.514 0.509 0.418 0.535 6 Paulus, J. 0.425 55.82% 55.10% 0.555 0.444 0.489 0.412 7 Gillet & Richard 2 0.273 66.68% 28.22% 0.397 0.393 0.178 0.269 8 Gillet & Richard 1 0.259 58.48% 26.11% 0.361 0.375 0.196 0.210

KOEN TANGHE COLLECTION
Rank	Participant	Total Average Classification F-measure	Total Overall Onset Precision	Total Overall Onset Recall	Total Overall Onset F-measure	BD Average F-measure	HH Average F-measure	SD Average F-measure	Runtime (s)	Machine

1 Yoshii, Goto, & Okuno 0.716 76.13% 69.16% 0.725 0.776 0.661 0.710 2 Tanghe, Degroeve, & De Baets 4 0.685 72.32% 75.83% 0.740 0.766 0.691 0.599 3 Tanghe, Degroeve, & De Baets 3 0.683 72.74% 75.65% 0.742 0.763 0.701 0.585 4 Tanghe, Degroeve, & De Baets 1 0.673 69.86% 77.12% 0.733 0.753 0.693 0.574 5 Gillet & Richard 2 0.630 81.60% 53.00% 0.643 0.774 0.517 0.599 6 Dittmar, C. 0.617 71.37% 67.78% 0.695 0.631 0.675 0.544 7 Paulus, J. 0.597 62.90% 75.47% 0.686 0.648 0.695 0.449 8 Gillet & Richard 1 0.544 76.11% 48.80% 0.595 0.715 0.479 0.436

MASATAKA GOTO COLLECTION (50 songs from RWC Music Database: RWC-MDB-P-2001)
Rank	Participant	Total Average Classification F-measure	Total Overall Onset Precision	Total Overall Onset Recall	Total Overall Onset F-measure	BD Average F-measure	HH Average F-measure	SD Average F-measure	Runtime (s)	Machine