Difference between revisions of "2015:Audio Downbeat Estimation Results"

From MIREX Wiki
(Created page with "= Submitted Algorithms = {|class="wikitable" style="text-align: left;" |+ Algorithms submitted to the Automatic Downbeat Estimation task ! width="80" | Submission code ...")
 
(→‎Results)
Line 80: Line 80:
 
! Precision
 
! Precision
 
! Recall
 
! Recall
 +
|-
 +
| DBDR2
 +
| 0.855
 +
| 0.866
 +
| 0.874
 +
|-
 +
| DBDR3
 +
| 0.847
 +
| 0.863
 +
| 0.862
 +
|-
 +
| FK2
 +
| 0.695
 +
| 0.691
 +
| 0.718
 
|-
 
|-
 
| FK3
 
| FK3
−
| 58.8
+
| 0.712
−
| 57.5
+
| 0.713
−
| 63.4
+
| 0.728
 
|-
 
|-
 
| FK4
 
| FK4
−
| 63.0
+
| 0.178
−
| 59.5
+
| 0.191
−
| 70.2
+
| 0.178
 
|-
 
|-
−
| KSH1
+
| FK6
−
| 19.4
+
| 0.643
−
| 21.1
+
| 0.616
−
| 20.8
+
| 0.697
 
|-
 
|-
−
| DBDR2*
 
−
| 85.5
 
−
| 86.6
 
−
| 87.4c
 
−
|-
 
−
| DBDR3
 
−
| 81.6
 
−
| 81.4
 
−
| 85.6
 
 
|}
 
|}
  
Line 115: Line 121:
 
! Precision
 
! Precision
 
! Recall
 
! Recall
 +
|-
 +
| DBDR2
 +
| 0.221
 +
| 0.195
 +
| 0.297
 +
|-
 +
| DBDR3
 +
| 0.216
 +
| 0.193
 +
| 0.289
 +
|-
 +
| FK2
 +
| 0.167
 +
| 0.145
 +
| 0.225
 
|-
 
|-
 
| FK3
 
| FK3
−
| 16.9
+
| 0.166
−
| 16.1
+
| 0.144
−
| 20.2
+
| 0.226
 
|-
 
|-
 
| FK4
 
| FK4
−
| 19.4
+
| 0.474
−
| 15.3
+
| 0.480
−
| 30.4
+
| 0.482
−
|-
 
−
| KSH1
 
−
| 40.0
 
−
| 39.9
 
−
| 41.4
 
−
|-
 
−
| DBDR2
 
−
| 18.4
 
−
| 15.6
 
−
| 25.9
 
 
|-
 
|-
−
| DBDR3
+
| FK6
−
| 20.0
+
| 0.192
−
| 16.9
+
| 0.156
−
| 28.2
+
| 0.282
 
|}
 
|}
  
Line 150: Line 161:
 
! Precision
 
! Precision
 
! Recall
 
! Recall
 +
|-
 +
| DBDR2
 +
| 0.472
 +
| 0.445
 +
| 0.579
 +
|-
 +
| DBDR3
 +
| 0.446
 +
| 0.409
 +
| 0.558
 +
|-
 +
| FK2
 +
| 0.295
 +
| 0.244
 +
| 0.420
 
|-
 
|-
 
| FK3
 
| FK3
−
| 19.7
+
| 0.281
−
| 15.8
+
| 0.237
−
| 28.9
+
| 0.392
 
|-
 
|-
 
| FK4
 
| FK4
−
| 24.0
+
| 0.142
−
| 18.2
+
| 0.145
−
| 38.7
+
| 0.151
 
|-
 
|-
−
| KSH1*
+
| FK6
−
| 77.5
+
| 0.258
−
| 75.9
+
| 0.210
−
| 81.9
+
| 0.391
−
|-
 
−
| DBDR2
 
−
| 44.8
 
−
| 40.0
 
−
| 55.3
 
−
|-
 
−
| DBDR3
 
−
| 44.8
 
−
| 39.5
 
−
| 56.5
 
 
|}
 
|}
  
Line 186: Line 202:
 
! Precision
 
! Precision
 
! Recall
 
! Recall
 +
|-
 +
| DBDR2
 +
| 0.415
 +
| 0.640
 +
| 0.307
 +
|-
 +
| DBDR3
 +
| 0.449
 +
| 0.693
 +
| 0.332
 +
|-
 +
| FK2
 +
| 0.201
 +
| 0.301
 +
| 0.151
 
|-
 
|-
 
| FK3
 
| FK3
−
| 53.5
+
| 0.169
−
| 86.6
+
| 0.252
−
| 39.3
+
| 0.127
 
|-
 
|-
 
| FK4
 
| FK4
−
| 51.2
+
| 0.233
−
| 74.5
+
| 0.382
−
| 39.4
+
| 0.176
 
|-
 
|-
−
| KSH1*
+
| FK6
−
| 85.4
+
| 0.467
−
| 88.2
+
| 0.698
−
| 84.1
+
| 0.351
−
|-
 
−
| DBDR2
 
−
| 43.5
 
−
| 65.5
 
−
| 32.5
 
−
|-
 
−
| DBDR3
 
−
| 41.5
 
−
| 62.5
 
−
| 31.0
 
 
|}
 
|}
  
Line 221: Line 242:
 
! Precision
 
! Precision
 
! Recall
 
! Recall
 +
|-
 +
| DBDR2
 +
| 0.691
 +
| 0.725
 +
| 0.676
 +
|-
 +
| DBDR3
 +
| 0.682
 +
| 0.705
 +
| 0.672
 +
|-
 +
| FK2
 +
| 0.846
 +
| 0.847
 +
| 0.845
 
|-
 
|-
 
| FK3
 
| FK3
−
| 21.5
+
| 0.755
−
| 26.2
+
| 0.756
−
| 19.1
+
| 0.755
 
|-
 
|-
 
| FK4
 
| FK4
−
| 64.0
+
| 0.120
−
| 64.2
+
| 0.232
−
| 63.9
+
| 0.096
 
|-
 
|-
−
| KSH1
+
| FK6
−
| 9.4
+
| 0.616
−
| 16.7
+
| 0.617
−
| 8.5
+
| 0.616
−
|-
 
−
| DBDR2
 
−
| 64.2
 
−
| 65.1
 
−
| 64.2
 
−
|-
 
−
| DBDR3
 
−
| 66.7
 
−
| 67.6
 
−
| 66.8
 
 
|}
 
|}
  

Revision as of 00:31, 22 September 2015

Submitted Algorithms

Algorithms submitted to the Automatic Downbeat Estimation task
Submission code Submission name Abstract Contributors
DBDR2 DB1_no_ballroom PDF Simon Durand, Juan Bello, Bertrand David, Gael Richard
DBDR3 DB2_no_beatles PDF Simon Durand, Juan Bello, Bertrand David, Gael Richard
FK2 mixed_no_ballroom PDF Florian Krebs, Sebastian Böck
FK3 mixed_no_beatles PDF Florian Krebs, Sebastian Böck
FK4 carnatic_tracker PDF Florian Krebs, Sebastian Böck
FK6 2014_tracker_new_ss PDF Florian Krebs, Sebastian Böck

Results

Results ballroom dataset
Algorithm F-Measure Precision Recall
DBDR2 0.756 0.797 0.765
DBDR3 0.793 0.833 0.801
FK2 0.556 0.567 0.564
FK3 0.622 0.631 0.632
FK4 0.175 0.197 0.171
FK6 0.683 0.669 0.712


Results beatles dataset
Algorithm F-Measure Precision Recall
DBDR2 0.855 0.866 0.874
DBDR3 0.847 0.863 0.862
FK2 0.695 0.691 0.718
FK3 0.712 0.713 0.728
FK4 0.178 0.191 0.178
FK6 0.643 0.616 0.697


Results carnatic dataset
Algorithm F-Measure Precision Recall
DBDR2 0.221 0.195 0.297
DBDR3 0.216 0.193 0.289
FK2 0.167 0.145 0.225
FK3 0.166 0.144 0.226
FK4 0.474 0.480 0.482
FK6 0.192 0.156 0.282


Results turkish dataset
Algorithm F-Measure Precision Recall
DBDR2 0.472 0.445 0.579
DBDR3 0.446 0.409 0.558
FK2 0.295 0.244 0.420
FK3 0.281 0.237 0.392
FK4 0.142 0.145 0.151
FK6 0.258 0.210 0.391


Results cretan dataset
Algorithm F-Measure Precision Recall
DBDR2 0.415 0.640 0.307
DBDR3 0.449 0.693 0.332
FK2 0.201 0.301 0.151
FK3 0.169 0.252 0.127
FK4 0.233 0.382 0.176
FK6 0.467 0.698 0.351


Results hjdb dataset
Algorithm F-Measure Precision Recall
DBDR2 0.691 0.725 0.676
DBDR3 0.682 0.705 0.672
FK2 0.846 0.847 0.845
FK3 0.755 0.756 0.755
FK4 0.120 0.232 0.096
FK6 0.616 0.617 0.616

*) Rows marked by an asterisk should be taken with care as in those cases overlapping test and training sets were used.

Runtime

All submissions except KSH1 finished the computations in less than 24 hours.

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2015:Audio_Downbeat_Estimation_Results&oldid=11257"