2019:Music Detection Results - Revision history

Blai Melendez-Catalan: /* Event-level Evaluation */

2019-10-30T13:55:58Z

‎Event-level Evaluation

Blai Melendez-Catalan: /* Task 2: Relative Music Loudness Estimation */

2019-10-30T13:27:00Z

‎Task 2: Relative Music Loudness Estimation

Blai Melendez-Catalan: /* Event-level Evaluation */

2019-10-30T13:26:01Z

‎Event-level Evaluation

Blai Melendez-Catalan: /* Event-level Evaluation */

2019-10-30T13:23:27Z

‎Event-level Evaluation

Blai Melendez-Catalan: /* Segment-level Evaluation */

2019-10-30T12:50:36Z

‎Segment-level Evaluation

Blai Melendez-Catalan: /* Segment-level Evaluation */

2019-10-30T12:43:33Z

‎Segment-level Evaluation

Blai Melendez-Catalan at 12:27, 30 October 2019

2019-10-30T12:27:16Z

Blai Melendez-Catalan: Created page with "==Introduction== These are the results for the 2018 running of the Music and/or Speech Detection tasks. For background information about this task set please refer to the 20..."

2019-10-30T10:37:36Z

Created page with "==Introduction== These are the results for the 2018 running of the Music and/or Speech Detection tasks. For background information about this task set please refer to the 20..."

New page

==Introduction==
These are the results for the 2018 running of the Music and/or Speech Detection tasks. For background information about this task set please refer to the [[2018:Music and/or Speech Detection]] page.

==General Legend==
{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Sub code
! width="80" style="text-align: center;" | Abstract
! width="440" | Contributors
|-
! MMG1
| [https://www.music-ir.org/mirex/abstracts/2018/MMG.pdf PDF] || Blai Meléndez-Catalán, Emilio Molina, Emilia Gómez
|-
! MMG2
| [https://www.music-ir.org/mirex/abstracts/2018/MMG.pdf PDF] || Blai Meléndez-Catalán, Emilio Molina, Emilia Gómez
|-
! MMG3
| [https://www.music-ir.org/mirex/abstracts/2018/MMG.pdf PDF] || Blai Meléndez-Catalán, Emilio Molina, Emilia Gómez
|}

==Statistics notation==

Accuracy = segment-level accuracy

<class>_P = segment-level precision for the class <class>

<class>_R = segment-level recall for the class <class>

<class>_F = segment-level F-measure for the class <class>

<class>_F_500_on = onset-only event-level F-measure (500 ms tolerance) for the class <class>

<class>_F_500_onoff = onset-offset event-level F-measure (500 ms tolerance) for the class <class>

<class>_F_1000_on = onset-only event-level F-measure (1000 ms tolerance) for the class <class>

<class>_F_1000_onoff = onset-offset event-level F-measure (1000 ms tolerance) for the class <class>

==Datasets description==

[https://www.music-ir.org/mirex/wiki/2018:Music_and/or_Speech_Detection#Evaluation_Dataset Dataset description]

==Task 1: Music Detection==

===Segment-level Evaluation===

{| border="1" cellspacing="0" style="text-align: left; width: 240px;"
|- style="background: yellow;"
! width="80" | Sub code
! width="80" style="text-align: center;" | Accuracy
! width="80" | Music_P
! width="80" | Music_R
! width="80" | Music_F
! width="80" | No-Music_P
! width="80" | No-Music_R
! width="80" | No-Music_F
|-
! MMG1
| 0.9049 || 0.9131 || 0.8865 || 0.8996 || 0.8978 || 0.9219 || 0.9097
|-
! MMG2
| 0.9049 || 0.9131 || 0.8865 || 0.8996 || 0.8978 || 0.9219 || 0.9097
|-
! MMG3
| 0.8506 || 0.967 || 0.7134 || 0.8211 || 0.7866 || 0.9775 || 0.8717
|}

===Event-level Evaluation===

{| border="1" cellspacing="0" style="text-align: left; width: 240px;"
|- style="background: yellow;"
! width="80" | Sub code
! width="80" style="text-align: center;" | Music_F_500_on
! width="80" | Music_F_500_onoff
! width="80" | Music_F_1000_on
! width="80" | Music_F_1000_onoff
|-
! MMG1
| 0.5177 || 0.2693 || 0.5813 || 0.3502
|-
! MMG2
| 0.5177 || 0.2693 || 0.5813 || 0.3502
|-
! MMG3
| 0.4403 || 0.1991 || 0.4973 || 0.2788
|}

==Task 2: Relative Music Loudness Estimation==

===Segment-level Evaluation===

{| border="1" cellspacing="0" style="text-align: left; width: 240px;"
|- style="background: yellow;"
! width="80" | Sub code
! width="80" style="text-align: center;" | Accuracy
! width="80" | Fg-Music_P
! width="80" | Fg-Music_R
! width="80" | Fg-Music_F
! width="80" | Bg-Music_P
! width="80" | Bg-Music_R
! width="80" | Bg-Music_F
! width="80" | No-Music_P
! width="80" | No-Music_R
! width="80" | No-Music_F
|-
! MMG1
| 0.8615 || 0.8025 || 0.774 || 0.788 || 0.8211 || 0.821 || 0.821 || 0.9026 || 0.9103 || 0.9064
|-
! MMG2
| 0.8615 || 0.8025 || 0.774 || 0.788 || 0.8211 || 0.821 || 0.821 || 0.9026 || 0.9103 || 0.9064
|-
! MMG3
| 0.8615 || 0.8025 || 0.774 || 0.788 || 0.8211 || 0.821 || 0.821 || 0.9026 || 0.9103 || 0.9064
|}

====Event-level Evaluation====

{| border="1" cellspacing="0" style="text-align: left; width: 240px;"
|- style="background: yellow;"
! width="80" | Sub code
! width="80" style="text-align: center;" | Fg-Music_F_500_on
! width="80" | Fg-Music_F_500_onoff
! width="80" | Fg-Music_F_1000_on
! width="80" | Fg-Music_F_1000_onoff
! width="80" | Bg-Music_F_500_on
! width="80" | Bg-Music_F_500_onoff
! width="80" | Bg-Music_F_1000_on
! width="80" | Bg-Music_F_1000_onoff
! width="80" | No-Music_F_500_on
! width="80" | No-Music_F_500_onoff
! width="80" | No-Music_F_1000_on
! width="80" | Speech_F_1000_onoff
|-
! MMG1
| 0.3298 || 0.1775 || 0.4106 || 0.2742 || 0.3853 || 0.1388 || 0.4463 || 0.2024 || 0.5254 || 0.3123 || 0.5927 || 0.3925
|-
! MMG2
| 0.3298 || 0.1775 || 0.4106 || 0.2742 || 0.3853 || 0.1388 || 0.4463 || 0.2024 || 0.5254 || 0.3123 || 0.5927 || 0.3925
|-
! MMG3
| 0.3298 || 0.1775 || 0.4106 || 0.2742 || 0.3853 || 0.1388 || 0.4463 || 0.2024 || 0.5254 || 0.3123 || 0.5927 || 0.3925
|}

@@ Line 117: / Line 117: @@
 ====Event-level Evaluation====
-====Onset 500 ms tolerance====
+=====Onset 500 ms tolerance=====
 {| border="1" cellspacing="0" style="text-align: left; width: 240px;"
@@ Line 136: / Line 136: @@
          |}
-====Onset-offset 500 ms tolerance====
+=====Onset-offset 500 ms tolerance=====
 {| border="1" cellspacing="0" style="text-align: left; width: 240px;"
@@ Line 155: / Line 155: @@
          |}
-====Onset 1000 ms tolerance====
+=====Onset 1000 ms tolerance=====
 {| border="1" cellspacing="0" style="text-align: left; width: 240px;"
@@ Line 174: / Line 174: @@
          |}
-====Onset-offset 1000 ms tolerance====
+=====Onset-offset 1000 ms tolerance=====
 {| border="1" cellspacing="0" style="text-align: left; width: 240px;"

@@ Line 116: / Line 116: @@
 ====Event-level Evaluation====
 {| border="1" cellspacing="0" style="text-align: left; width: 240px;"
@@ Line 133: / Line 135: @@
 	| 0.4852 || 0.4916 || 0.5627
          |}
 {| border="1" cellspacing="0" style="text-align: left; width: 240px;"
@@ Line 150: / Line 154: @@
 	| 0.3136 || 0.2287 || 0.38
          |}
 {| border="1" cellspacing="0" style="text-align: left; width: 240px;"
@@ Line 167: / Line 173: @@
 	| 0.5493 || 0.5414 || 0.6213
          |}
 {| border="1" cellspacing="0" style="text-align: left; width: 240px;"

@@ Line 77: / Line 77: @@
          |-
          ! MMG1
-	| 0.5177 || 0.2693 || 0.5813 || 0.3502
+	| 0.5176 || 0.2456 || 0.5664 || 0.3108
 	|-
          ! MMG2
-	| 0.5177 || 0.2693 || 0.5813 || 0.3502
+	| 0.5486 || 0.2748 || 0.5961 || 0.342
          |-
          ! MMG3
-	| 0.4403 || 0.1991 || 0.4973 || 0.2788
+	| 0.6456 || 0.383 || 0.6903 || 0.4538
          |}
 ==Task 2: Relative Music Loudness Estimation==

@@ Line 121: / Line 121: @@
 	! width="80" | Sub code
 	! width="80" style="text-align: center;" | Fg-Music_F_500_on
 	! width="80" | Bg-Music_F_500_on
 	! width="80" | Bg-Music_F_500_onoff
 	! width="80" | Bg-Music_F_1000_on
 	! width="80" | Bg-Music_F_1000_onoff
-	! width="80" | No-Music_F_500_on
+	! width="80" | No-Music_F_1000_onoff
 	|-
 	! MMG1
-	| 0.3298 || 0.1775 || 0.4106 || 0.2742 || 0.3853 || 0.1388 || 0.4463 || 0.2024 || 0.5254 || 0.3123 || 0.5927 || 0.3925
+	| 0.1584 || 0.1236 || 0.3283
 	|-
 	! MMG2
-	| 0.3298 || 0.1775 || 0.4106 || 0.2742 || 0.3853 || 0.1388 || 0.4463 || 0.2024 || 0.5254 || 0.3123 || 0.5927 || 0.3925
+	| 0.1644 || 0.1395 || 0.4002
 	|-
 	! MMG3
-	| 0.3298 || 0.1775 || 0.4106 || 0.2742 || 0.3853 || 0.1388 || 0.4463 || 0.2024 || 0.5254 || 0.3123 || 0.5927 || 0.3925
+	| 0.4038 || 0.2929 || 0.4556
          |}

@@ Line 106: / Line 106: @@
 	|-
          ! MMG1
-	| 0.8615 || 0.8025 || 0.774 || 0.788 || 0.8211 || 0.821 || 0.821 || 0.9026 || 0.9103 || 0.9064
+	| 0.8152 || 0.7874 || 0.7392 || 0.7626 || 0.7941 || 0.7647 || 0.7791 || 0.8398 || 0.8834 || 0.861
 	|-
          ! MMG2
-	| 0.8615 || 0.8025 || 0.774 || 0.788 || 0.8211 || 0.821 || 0.821 || 0.9026 || 0.9103 || 0.9064
+	| 0.8414 || 0.8741 || 0.7055 || 0.7808 || 0.8035 || 0.8154 || 0.8095 || 0.8648 || 0.9079 || 0.8858
 	|-
          ! MMG3
-	| 0.8615 || 0.8025 || 0.774 || 0.788 || 0.8211 || 0.821 || 0.821 || 0.9026 || 0.9103 || 0.9064
+	| 0.8749 || 0.8375 || 0.8291 || 0.8333 || 0.8238 || 0.8896 || 0.8554 || 0.9387 || 0.8773 || 0.907
          |}

@@ Line 57: / Line 57: @@
          |-
          ! MMG1
-	| 0.9049 || 0.9131 || 0.8865 || 0.8996 || 0.8978 || 0.9219 || 0.9097
+	| 0.8713 || 0.9056 || 0.8513 || 0.8776 || 0.8354 || 0.8953 || 0.8643
 	|-
          ! MMG2
-	| 0.9049 || 0.9131 || 0.8865 || 0.8996 || 0.8978 || 0.9219 || 0.9097
+	| 0.8928 || 0.9186 || 0.8803 || 0.899 || 0.8648 || 0.9079 || 0.8858
          |-
          ! MMG3
-	| 0.8506 || 0.967 || 0.7134 || 0.8211 || 0.7866 || 0.9775 || 0.8717
+	| 0.9178 || 0.9026 || 0.9511 || 0.9262 || 0.9381 || 0.8787 || 0.9074
          |}

@@ Line 13: / Line 13: @@
          |-
 	! MMG2
-         | [https://www.music-ir.org/mirex/abstracts/2018/MMG.pdf PDF] || Blai Meléndez-Catalán, Emilio Molina, Emilia Gómez
+         | [https://www.music-ir.org/mirex/abstracts/2018/MGM.pdf PDF] || Blai Meléndez-Catalán, Emilio Molina, Emilia Gómez
          |-
 	! MMG3
-         | [https://www.music-ir.org/mirex/abstracts/2018/MMG.pdf PDF] || Blai Meléndez-Catalán, Emilio Molina, Emilia Gómez
+         | [https://www.music-ir.org/mirex/abstracts/2018/MGM.pdf PDF] || Blai Meléndez-Catalán, Emilio Molina, Emilia Gómez
          |}