2015:Audio Downbeat Estimation Results - Revision history

Florian: /* Results */

2015-10-10T09:27:08Z

‎Results

Florian: /* Results */

2015-10-10T09:26:25Z

‎Results

Florian: /* Runtime */

2015-10-07T05:23:24Z

‎Runtime

Florian: /* Submitted Algorithms */

2015-10-07T05:20:54Z

‎Submitted Algorithms

Florian: /* Submitted Algorithms */

2015-10-06T11:00:56Z

‎Submitted Algorithms

Florian at 07:40, 6 October 2015

2015-10-06T07:40:02Z

Florian at 06:37, 22 September 2015

2015-09-22T06:37:11Z

Florian: /* Results */

2015-09-22T06:35:18Z

‎Results

Florian: /* Results */

2015-09-22T06:31:48Z

‎Results

Show changes

Florian: Created page with "= Submitted Algorithms = {|class="wikitable" style="text-align: left;" |+ Algorithms submitted to the Automatic Downbeat Estimation task ! width="80" | Submission code ..."

2015-09-22T06:28:29Z

Created page with "= Submitted Algorithms = {|class="wikitable" style="text-align: left;" |+ Algorithms submitted to the Automatic Downbeat Estimation task ! width="80" | Submission code ..."

New page

= Submitted Algorithms =

{|class="wikitable" style="text-align: left;"
|+ Algorithms submitted to the Automatic Downbeat Estimation task
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract
! width="440" | Contributors
|-
!
!
|-
! DBDR2
| DB1_no_ballroom || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/DBDR2.pdf PDF] || Simon Durand, Juan Bello, Bertrand David, Gael Richard
|-
! DBDR3
| DB2_no_beatles || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/DBDR3.pdf PDF] || Simon Durand, Juan Bello, Bertrand David, Gael Richard
|-
! FK2
| mixed_no_ballroom || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK3.pdf PDF] || Florian Krebs, Sebastian Böck
|-
! FK3
| mixed_no_beatles || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK4.pdf PDF] || Florian Krebs, Sebastian Böck
|-
! FK4
| carnatic_tracker || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/KSH1.pdf PDF] || Florian Krebs, Sebastian Böck
|-
! FK6
| 2014_tracker_new_ss || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/KSH1.pdf PDF] || Florian Krebs, Sebastian Böck
|-
|}

= Results =

{| class="wikitable sortable" border="1"
|+ Results ballroom dataset
|-
! Algorithm
! F-Measure
! Precision
! Recall
|-
| DBDR2
| 0.756
| 0.797
| 0.765
|-
| DBDR3
| 0.793
| 0.833
| 0.801
|-
| FK2
| 0.556
| 0.567
| 0.564
|-
| FK3
| 0.622
| 0.631
| 0.632
|-
| FK4
| 0.175
| 0.197
| 0.171
|-
| FK6
| 0.683
| 0.669
| 0.712
|}

{| class="wikitable sortable" border="1"
|+ Results beatles dataset
|-
! Algorithm
! F-Measure
! Precision
! Recall
|-
| FK3
| 58.8
| 57.5
| 63.4
|-
| FK4
| 63.0
| 59.5
| 70.2
|-
| KSH1
| 19.4
| 21.1
| 20.8
|-
| DBDR2*
| 85.5
| 86.6
| 87.4c
|-
| DBDR3
| 81.6
| 81.4
| 85.6
|}

{| class="wikitable sortable" border="1"
|+ Results carnatic dataset
|-
! Algorithm
! F-Measure
! Precision
! Recall
|-
| FK3
| 16.9
| 16.1
| 20.2
|-
| FK4
| 19.4
| 15.3
| 30.4
|-
| KSH1
| 40.0
| 39.9
| 41.4
|-
| DBDR2
| 18.4
| 15.6
| 25.9
|-
| DBDR3
| 20.0
| 16.9
| 28.2
|}

{| class="wikitable sortable" border="1"
|+ Results turkish dataset
|-
! Algorithm
! F-Measure
! Precision
! Recall
|-
| FK3
| 19.7
| 15.8
| 28.9
|-
| FK4
| 24.0
| 18.2
| 38.7
|-
| KSH1*
| 77.5
| 75.9
| 81.9
|-
| DBDR2
| 44.8
| 40.0
| 55.3
|-
| DBDR3
| 44.8
| 39.5
| 56.5
|}

{| class="wikitable sortable" border="1"

|+ Results cretan dataset
|-
! Algorithm
! F-Measure
! Precision
! Recall
|-
| FK3
| 53.5
| 86.6
| 39.3
|-
| FK4
| 51.2
| 74.5
| 39.4
|-
| KSH1*
| 85.4
| 88.2
| 84.1
|-
| DBDR2
| 43.5
| 65.5
| 32.5
|-
| DBDR3
| 41.5
| 62.5
| 31.0
|}

{| class="wikitable sortable" border="1"
|+ Results hjdb dataset
|-
! Algorithm
! F-Measure
! Precision
! Recall
|-
| FK3
| 21.5
| 26.2
| 19.1
|-
| FK4
| 64.0
| 64.2
| 63.9
|-
| KSH1
| 9.4
| 16.7
| 8.5
|-
| DBDR2
| 64.2
| 65.1
| 64.2
|-
| DBDR3
| 66.7
| 67.6
| 66.8
|}

<nowiki>*</nowiki>) Rows marked by an asterisk should be taken with care as in those cases overlapping test and training sets were used.

= Runtime =
All submissions except KSH1 finished the computations in less than 24 hours.

@@ Line 46: / Line 46: @@
 | 0.770
 |-
-| DBDR3
+| DBDR3*
 | 0.802
 | 0.838
@@ Line 56: / Line 56: @@
 | 0.507
 |-
-| FK3
+| FK3*
 | 0.595
 | 0.598
@@ Line 66: / Line 66: @@
 | 0.175
 |-
-| FK6
+| FK6*
 | 0.756
 | 0.743
@@ Line 80: / Line 80: @@
 ! Recall
 |-
-| DBDR2
+| DBDR2*
 | 0.855
 | 0.866
@@ Line 90: / Line 90: @@
 | 0.862
 |-
-| FK2
+| FK2*
 | 0.713
 | 0.708

@@ Line 42: / Line 42: @@
 |-
 | DBDR2
-| 0.756
+| 0.763
-| 0.797
+| 0.807
-| 0.765
+| 0.770
 |-
-| DBDR3*
+| DBDR3
-| 0.793
+| 0.802
-| 0.833
+| 0.838
-| 0.801
+| 0.814
 |-
 | FK2
-| 0.557
+| 0.503
-| 0.566
+| 0.515
-| 0.568
+| 0.507
 |-
-| FK3*
+| FK3
-| 0.631
+| 0.595
-| 0.642
+| 0.598
-| 0.638
+| 0.606
 |-
 | FK4
 | 0.175
 |-
-| FK6*
+| FK6
-| 0.726
+| 0.756
 | 0.743
 |}
 {| class="wikitable sortable" border="1"
@@ Line 81: / Line 80: @@
 ! Recall
 |-
-| DBDR2*
+| DBDR2
 | 0.855
 | 0.866
@@ Line 91: / Line 90: @@
 | 0.862
 |-
-| FK2*
+| FK2
 | 0.713
 | 0.708
@@ Line 97: / Line 96: @@
 |-
 | FK3
-| 0.718
+| 0.709
-| 0.715
+| 0.706
-| 0.739
+| 0.730
 |-
 | FK4
@@ Line 107: / Line 106: @@
 |-
 | FK6
-| 0.665
+| 0.642
-| 0.662
+| 0.616
-| 0.687
+| 0.694
 |-
 |}
@@ Line 138: / Line 137: @@
 |-
 | FK3
-| 0.161
+| 0.166
-| 0.140
+| 0.144
-| 0.216
+| 0.225
 |-
 | FK4
@@ Line 148: / Line 147: @@
 |-
 | FK6
-| 0.173
+| 0.197
-| 0.150
+| 0.160
-| 0.232
+| 0.285
 |}
@@ Line 178: / Line 177: @@
 |-
 | FK3
-| 0.290
+| 0.298
-| 0.246
+| 0.249
-| 0.400
+| 0.423
 |-
 | FK4
@@ Line 188: / Line 187: @@
 |-
 | FK6
-| 0.287
+| 0.284
-| 0.239
+| 0.232
-| 0.414
+| 0.426
 |}
@@ Line 219: / Line 218: @@
 |-
 | FK3
-| 0.152
+| 0.167
-| 0.228
+| 0.249
-| 0.114
+| 0.125
 |-
 | FK4
@@ Line 229: / Line 228: @@
 |-
 | FK6
-| 0.423
+| 0.529
-| 0.634
+| 0.791
-| 0.317
+| 0.398
 |}
@@ Line 259: / Line 258: @@
 |-
 | FK3
-| 0.791
+| 0.824
-| 0.791
+| 0.824
-| 0.790
+| 0.823
 |-
 | FK4
@@ Line 269: / Line 268: @@
 |-
 | FK6
-| 0.706
+| 0.626
-| 0.706
+| 0.627
-| 0.706
+| 0.625
 |}

← Older revision		Revision as of 05:23, 7 October 2015
Line 277:		Line 277:

	= Runtime =		= Runtime =
−	All submissions ~~except KSH1~~ finished the computations in less than 24 hours.	+	All submissions finished the computations in less than 24 hours.

@@ Line 52: / Line 52: @@
 |-
 | FK2
-| 0.556
+| 0.557
-| 0.567
+| 0.566
-| 0.564
+| 0.568
 |-
 | FK3*
 | 0.631
-| 0.632
+| 0.642
 |-
 | FK4
@@ Line 67: / Line 67: @@
 |-
 | FK6*
-| 0.683
+| 0.726
-| 0.669
+| 0.721
-| 0.712
+| 0.743
 |}
@@ Line 92: / Line 92: @@
 |-
 | FK2*
-| 0.695
+| 0.713
-| 0.691
+| 0.708
-| 0.718
+| 0.738
 |-
 | FK3
-| 0.712
+| 0.718
-| 0.713
+| 0.715
-| 0.728
+| 0.739
 |-
 | FK4
@@ Line 107: / Line 107: @@
 |-
 | FK6
-| 0.643
+| 0.665
-| 0.616
+| 0.662
-| 0.697
+| 0.687
 |-
 |}
@@ Line 133: / Line 133: @@
 |-
 | FK2
-| 0.167
+| 0.154
-| 0.145
+| 0.133
-| 0.225
+| 0.205
 |-
 | FK3
-| 0.166
+| 0.161
-| 0.144
+| 0.140
-| 0.226
+| 0.216
 |-
 | FK4
@@ Line 148: / Line 148: @@
 |-
 | FK6
-| 0.192
+| 0.173
-| 0.156
+| 0.150
-| 0.282
+| 0.232
 |}
@@ Line 173: / Line 173: @@
 |-
 | FK2
-| 0.295
+| 0.289
-| 0.244
+| 0.247
-| 0.420
+| 0.393
 |-
 | FK3
-| 0.281
+| 0.290
-| 0.237
+| 0.246
-| 0.392
+| 0.400
 |-
 | FK4
@@ Line 188: / Line 188: @@
 |-
 | FK6
-| 0.258
+| 0.287
-| 0.210
+| 0.239
-| 0.391
+| 0.414
 |}
@@ Line 214: / Line 214: @@
 |-
 | FK2
 | 0.151
 |-
 | FK3
-| 0.169
+| 0.152
-| 0.252
+| 0.228
-| 0.127
+| 0.114
 |-
 | FK4
@@ Line 229: / Line 229: @@
 |-
 | FK6
-| 0.467
+| 0.423
-| 0.698
+| 0.634
-| 0.351
+| 0.317
 |}
@@ Line 254: / Line 254: @@
 |-
 | FK2
-| 0.846
+| 0.794
-| 0.847
+| 0.795
-| 0.845
+| 0.794
 |-
 | FK3
-| 0.755
+| 0.791
-| 0.756
+| 0.791
-| 0.755
+| 0.790
 |-
 | FK4
@@ Line 269: / Line 269: @@
 |-
 | FK6
-| 0.616
+| 0.706
-| 0.617
+| 0.706
-| 0.616
+| 0.706
 |}

@@ Line 27: / Line 27: @@
          |-
          ! FK6
-	| 2014_tracker_new_ss  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK6.pdf PDF] || Florian Krebs, Sebastian Böck
+	| 2014_FK4_new_ss  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK6.pdf PDF] || Florian Krebs, Sebastian Böck
          |-
 |}

@@ Line 18: / Line 18: @@
          |-
 	! FK2
-	| mixed_no_ballroom  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK3.pdf PDF] || Florian Krebs, Sebastian Böck
+	| mixed_no_ballroom  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK2.pdf PDF] || Florian Krebs, Sebastian Böck
          |-
 	! FK3
-	| mixed_no_beatles  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK4.pdf PDF] || Florian Krebs, Sebastian Böck
+	| mixed_no_beatles  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK3.pdf PDF] || Florian Krebs, Sebastian Böck
          |-
 	! FK4
-	| carnatic_tracker  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/KSH1.pdf PDF] || Florian Krebs, Sebastian Böck
+	| carnatic_tracker  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK4.pdf PDF] || Florian Krebs, Sebastian Böck
          |-
          ! FK6
-	| 2014_tracker_new_ss  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/KSH1.pdf PDF] || Florian Krebs, Sebastian Böck
+	| 2014_tracker_new_ss  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/FK6.pdf PDF] || Florian Krebs, Sebastian Böck
          |-
 |}