Difference between revisions of "2015:Audio Downbeat Estimation Results"

From MIREX Wiki
(→‎Results)
(→‎Results)
 
(6 intermediate revisions by the same user not shown)
Line 18: Line 18:
 
         |-
 
         |-
 
! FK2
 
! FK2
−
| mixed_no_ballroom  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK3.pdf PDF] || Florian Krebs, Sebastian Böck
+
| mixed_no_ballroom  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK2.pdf PDF] || Florian Krebs, Sebastian Böck
 
         |-
 
         |-
 
! FK3
 
! FK3
−
| mixed_no_beatles  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK4.pdf PDF] || Florian Krebs, Sebastian Böck
+
| mixed_no_beatles  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK3.pdf PDF] || Florian Krebs, Sebastian Böck
 
         |-
 
         |-
 
! FK4
 
! FK4
−
| carnatic_tracker  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/KSH1.pdf PDF] || Florian Krebs, Sebastian Böck
+
| carnatic_tracker  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK4.pdf PDF] || Florian Krebs, Sebastian Böck
 
         |-
 
         |-
 
         ! FK6
 
         ! FK6
−
| 2014_tracker_new_ss ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/KSH1.pdf PDF] || Florian Krebs, Sebastian Böck
+
| 2014_FK4_new_ss ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK6.pdf PDF] || Florian Krebs, Sebastian Böck
 
         |-
 
         |-
 
|}
 
|}
Line 42: Line 42:
 
|-
 
|-
 
| DBDR2
 
| DBDR2
−
| 0.756
+
| 0.763
−
| 0.797
+
| 0.807
−
| 0.765
+
| 0.770
 
|-
 
|-
−
| DBDR3
+
| DBDR3*
−
| 0.793
+
| 0.802
−
| 0.833
+
| 0.838
−
| 0.801
+
| 0.814
 
|-
 
|-
 
| FK2
 
| FK2
−
| 0.556
+
| 0.503
−
| 0.567
+
| 0.515
−
| 0.564
+
| 0.507
 
|-
 
|-
−
| FK3
+
| FK3*
−
| 0.622
+
| 0.595
−
| 0.631
+
| 0.598
−
| 0.632
+
| 0.606
 
|-
 
|-
 
| FK4
 
| FK4
 +
| 0.179
 +
| 0.204
 
| 0.175  
 
| 0.175  
−
| 0.197
 
−
| 0.171
 
 
|-
 
|-
−
| FK6
+
| FK6*
−
| 0.683
+
| 0.756
−
| 0.669
+
| 0.743
−
| 0.712
+
| 0.778
 
|}
 
|}
−
 
  
 
{| class="wikitable sortable" border="1"
 
{| class="wikitable sortable" border="1"
Line 91: Line 90:
 
| 0.862  
 
| 0.862  
 
|-
 
|-
−
| FK2
+
| FK2*
−
| 0.695
+
| 0.713
−
| 0.691
+
| 0.708
−
| 0.718
+
| 0.738
 
|-
 
|-
 
| FK3
 
| FK3
−
| 0.712
+
| 0.709
−
| 0.713
+
| 0.706
−
| 0.728
+
| 0.730
 
|-
 
|-
 
| FK4
 
| FK4
Line 107: Line 106:
 
|-
 
|-
 
| FK6
 
| FK6
−
| 0.643
+
| 0.642
 
| 0.616  
 
| 0.616  
−
| 0.697
+
| 0.694
 
|-
 
|-
 
|}
 
|}
Line 133: Line 132:
 
|-
 
|-
 
| FK2
 
| FK2
−
| 0.167
+
| 0.154
−
| 0.145
+
| 0.133
−
| 0.225
+
| 0.205
 
|-
 
|-
 
| FK3
 
| FK3
 
| 0.166  
 
| 0.166  
 
| 0.144  
 
| 0.144  
−
| 0.226
+
| 0.225
 
|-
 
|-
 
| FK4
 
| FK4
Line 148: Line 147:
 
|-
 
|-
 
| FK6
 
| FK6
−
| 0.192
+
| 0.197
−
| 0.156
+
| 0.160
−
| 0.282
+
| 0.285
 
|}
 
|}
  
Line 173: Line 172:
 
|-
 
|-
 
| FK2
 
| FK2
−
| 0.295
+
| 0.289
−
| 0.244
+
| 0.247
−
| 0.420
+
| 0.393
 
|-
 
|-
 
| FK3
 
| FK3
−
| 0.281
+
| 0.298
−
| 0.237
+
| 0.249
−
| 0.392
+
| 0.423
 
|-
 
|-
 
| FK4
 
| FK4
Line 188: Line 187:
 
|-
 
|-
 
| FK6
 
| FK6
−
| 0.258
+
| 0.284
−
| 0.210
+
| 0.232
−
| 0.391
+
| 0.426
 
|}
 
|}
  
Line 214: Line 213:
 
|-
 
|-
 
| FK2
 
| FK2
−
| 0.201
 
−
| 0.301
 
 
| 0.151  
 
| 0.151  
 +
| 0.226
 +
| 0.113
 
|-
 
|-
 
| FK3
 
| FK3
−
| 0.169
+
| 0.167
−
| 0.252
+
| 0.249
−
| 0.127
+
| 0.125
 
|-
 
|-
 
| FK4
 
| FK4
Line 229: Line 228:
 
|-
 
|-
 
| FK6
 
| FK6
−
| 0.467
+
| 0.529
−
| 0.698
+
| 0.791
−
| 0.351
+
| 0.398
 
|}
 
|}
  
Line 254: Line 253:
 
|-
 
|-
 
| FK2
 
| FK2
−
| 0.846
+
| 0.794
−
| 0.847
+
| 0.795
−
| 0.845
+
| 0.794
 
|-
 
|-
 
| FK3
 
| FK3
−
| 0.755
+
| 0.824
−
| 0.756
+
| 0.824
−
| 0.755
+
| 0.823
 
|-
 
|-
 
| FK4
 
| FK4
Line 269: Line 268:
 
|-
 
|-
 
| FK6
 
| FK6
−
| 0.616
+
| 0.626
−
| 0.617
+
| 0.627
−
| 0.616
+
| 0.625
 
|}
 
|}
  
Line 277: Line 276:
  
 
= Runtime =
 
= Runtime =
−
All submissions except KSH1 finished the computations in less than 24 hours.
+
All submissions finished the computations in less than 24 hours.

Latest revision as of 03:27, 10 October 2015

Submitted Algorithms

Algorithms submitted to the Automatic Downbeat Estimation task
Submission code Submission name Abstract Contributors
DBDR2 DB1_no_ballroom PDF Simon Durand, Juan Bello, Bertrand David, Gael Richard
DBDR3 DB2_no_beatles PDF Simon Durand, Juan Bello, Bertrand David, Gael Richard
FK2 mixed_no_ballroom PDF Florian Krebs, Sebastian Böck
FK3 mixed_no_beatles PDF Florian Krebs, Sebastian Böck
FK4 carnatic_tracker PDF Florian Krebs, Sebastian Böck
FK6 2014_FK4_new_ss PDF Florian Krebs, Sebastian Böck

Results

Results ballroom dataset
Algorithm F-Measure Precision Recall
DBDR2 0.763 0.807 0.770
DBDR3* 0.802 0.838 0.814
FK2 0.503 0.515 0.507
FK3* 0.595 0.598 0.606
FK4 0.179 0.204 0.175
FK6* 0.756 0.743 0.778
Results beatles dataset
Algorithm F-Measure Precision Recall
DBDR2* 0.855 0.866 0.874
DBDR3 0.847 0.863 0.862
FK2* 0.713 0.708 0.738
FK3 0.709 0.706 0.730
FK4 0.178 0.191 0.178
FK6 0.642 0.616 0.694


Results carnatic dataset
Algorithm F-Measure Precision Recall
DBDR2 0.221 0.195 0.297
DBDR3 0.216 0.193 0.289
FK2 0.154 0.133 0.205
FK3 0.166 0.144 0.225
FK4 0.474 0.480 0.482
FK6 0.197 0.160 0.285


Results turkish dataset
Algorithm F-Measure Precision Recall
DBDR2 0.472 0.445 0.579
DBDR3 0.446 0.409 0.558
FK2 0.289 0.247 0.393
FK3 0.298 0.249 0.423
FK4 0.142 0.145 0.151
FK6 0.284 0.232 0.426


Results cretan dataset
Algorithm F-Measure Precision Recall
DBDR2 0.415 0.640 0.307
DBDR3 0.449 0.693 0.332
FK2 0.151 0.226 0.113
FK3 0.167 0.249 0.125
FK4 0.233 0.382 0.176
FK6 0.529 0.791 0.398


Results hjdb dataset
Algorithm F-Measure Precision Recall
DBDR2 0.691 0.725 0.676
DBDR3 0.682 0.705 0.672
FK2 0.794 0.795 0.794
FK3 0.824 0.824 0.823
FK4 0.120 0.232 0.096
FK6 0.626 0.627 0.625

*) Rows marked by an asterisk should be taken with care as in those cases overlapping test and training sets were used.

Runtime

All submissions finished the computations in less than 24 hours.

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2015:Audio_Downbeat_Estimation_Results&oldid=11275"