Difference between revisions of "2019:Music Detection Results"

From MIREX Wiki
(→‎Event-level Evaluation)
 
(5 intermediate revisions by the same user not shown)
Line 57: Line 57:
 
         |-
 
         |-
 
         ! MMG1
 
         ! MMG1
−
| 0.9049 || 0.9131 || 0.8865 || 0.8996 || 0.8978 || 0.9219 || 0.9097
+
| 0.8713 || 0.9056 || 0.8513 || 0.8776 || 0.8354 || 0.8953 || 0.8643
 
|-
 
|-
 
         ! MMG2
 
         ! MMG2
−
| 0.9049 || 0.9131 || 0.8865 || 0.8996 || 0.8978 || 0.9219 || 0.9097
+
| 0.8928 || 0.9186 || 0.8803 || 0.899 || 0.8648 || 0.9079 || 0.8858
 
         |-
 
         |-
 
         ! MMG3
 
         ! MMG3
−
| 0.8506 || 0.967 || 0.7134 || 0.8211 || 0.7866 || 0.9775 || 0.8717
+
| 0.9178 || 0.9026 || 0.9511 || 0.9262 || 0.9381 || 0.8787 || 0.9074
 
         |}
 
         |}
  
Line 77: Line 77:
 
         |-
 
         |-
 
         ! MMG1
 
         ! MMG1
−
| 0.5177 || 0.2693 || 0.5813 || 0.3502
+
| 0.5176 || 0.2456 || 0.5664 || 0.3108
 
|-
 
|-
 
         ! MMG2
 
         ! MMG2
−
| 0.5177 || 0.2693 || 0.5813 || 0.3502
+
| 0.5486 || 0.2748 || 0.5961 || 0.342
 
         |-
 
         |-
 
         ! MMG3
 
         ! MMG3
−
| 0.4403 || 0.1991 || 0.4973 || 0.2788
+
| 0.6456 || 0.383 || 0.6903 || 0.4538
 
         |}
 
         |}
−
 
  
 
==Task 2: Relative Music Loudness Estimation==
 
==Task 2: Relative Music Loudness Estimation==
Line 106: Line 105:
 
|-
 
|-
 
         ! MMG1
 
         ! MMG1
−
| 0.8615 || 0.8025 || 0.774 || 0.788 || 0.8211 || 0.821 || 0.821 || 0.9026 || 0.9103 || 0.9064
+
| 0.8152 || 0.7874 || 0.7392 || 0.7626 || 0.7941 || 0.7647 || 0.7791 || 0.8398 || 0.8834 || 0.861
 
|-
 
|-
 
         ! MMG2
 
         ! MMG2
−
| 0.8615 || 0.8025 || 0.774 || 0.788 || 0.8211 || 0.821 || 0.821 || 0.9026 || 0.9103 || 0.9064
+
| 0.8414 || 0.8741 || 0.7055 || 0.7808 || 0.8035 || 0.8154 || 0.8095 || 0.8648 || 0.9079 || 0.8858
 
|-
 
|-
 
         ! MMG3
 
         ! MMG3
−
| 0.8615 || 0.8025 || 0.774 || 0.788 || 0.8211 || 0.821 || 0.821 || 0.9026 || 0.9103 || 0.9064
+
| 0.8749 || 0.8375 || 0.8291 || 0.8333 || 0.8238 || 0.8896 || 0.8554 || 0.9387 || 0.8773 || 0.907
 
         |}
 
         |}
  
 
====Event-level Evaluation====
 
====Event-level Evaluation====
 +
 +
=====Onset 500 ms tolerance=====
  
 
{| border="1" cellspacing="0" style="text-align: left; width: 240px;"
 
{| border="1" cellspacing="0" style="text-align: left; width: 240px;"
Line 121: Line 122:
 
! width="80" | Sub code  
 
! width="80" | Sub code  
 
! width="80" style="text-align: center;" | Fg-Music_F_500_on
 
! width="80" style="text-align: center;" | Fg-Music_F_500_on
−
! width="80" | Fg-Music_F_500_onoff
 
−
! width="80" | Fg-Music_F_1000_on
 
−
! width="80" | Fg-Music_F_1000_onoff
 
 
! width="80" | Bg-Music_F_500_on
 
! width="80" | Bg-Music_F_500_on
 +
! width="80" | No-Music_F_500_on
 +
|-
 +
! MMG1
 +
| 0.2413 || 0.3112 || 0.4729
 +
|-
 +
! MMG2
 +
| 0.2418 || 0.3424 || 0.5178
 +
|-
 +
! MMG3
 +
| 0.4852 || 0.4916 || 0.5627
 +
        |}
 +
 +
=====Onset-offset 500 ms tolerance=====
 +
 +
{| border="1" cellspacing="0" style="text-align: left; width: 240px;"
 +
|- style="background: yellow;"
 +
! width="80" | Sub code
 +
! width="80" style="text-align: center;" | Fg-Music_F_500_onoff
 
! width="80" | Bg-Music_F_500_onoff
 
! width="80" | Bg-Music_F_500_onoff
 +
! width="80" | No-Music_F_500_onoff
 +
|-
 +
! MMG1
 +
| 0.1045 || 0.0777 || 0.2646
 +
|-
 +
! MMG2
 +
| 0.1172 || 0.0934 || 0.3329
 +
|-
 +
! MMG3
 +
| 0.3136 || 0.2287 || 0.38
 +
        |}
 +
 +
=====Onset 1000 ms tolerance=====
 +
 +
{| border="1" cellspacing="0" style="text-align: left; width: 240px;"
 +
|- style="background: yellow;"
 +
! width="80" | Sub code
 +
! width="80" style="text-align: center;" | Fg-Music_F_1000_on
 
! width="80" | Bg-Music_F_1000_on
 
! width="80" | Bg-Music_F_1000_on
 +
! width="80" | No-Music_F_1000_on
 +
|-
 +
! MMG1
 +
| 0.3078 || 0.3629 || 0.525
 +
|-
 +
! MMG2
 +
| 0.3033 || 0.3932 || 0.5748
 +
|-
 +
! MMG3
 +
| 0.5493 || 0.5414 || 0.6213
 +
        |}
 +
 +
=====Onset-offset 1000 ms tolerance=====
 +
 +
{| border="1" cellspacing="0" style="text-align: left; width: 240px;"
 +
|- style="background: yellow;"
 +
! width="80" | Sub code
 +
! width="80" style="text-align: center;" | Fg-Music_F_1000_onoff
 
! width="80" | Bg-Music_F_1000_onoff
 
! width="80" | Bg-Music_F_1000_onoff
−
! width="80" | No-Music_F_500_on
+
! width="80" | No-Music_F_1000_onoff
−
! width="80" | No-Music_F_500_onoff
 
−
! width="80" | No-Music_F_1000_on
 
−
! width="80" | Speech_F_1000_onoff
 
 
|-
 
|-
 
! MMG1
 
! MMG1
−
| 0.3298 || 0.1775 || 0.4106 || 0.2742 || 0.3853 || 0.1388 || 0.4463 || 0.2024 || 0.5254 || 0.3123 || 0.5927 || 0.3925
+
| 0.1584 || 0.1236 || 0.3283
 
|-
 
|-
 
! MMG2
 
! MMG2
−
| 0.3298 || 0.1775 || 0.4106 || 0.2742 || 0.3853 || 0.1388 || 0.4463 || 0.2024 || 0.5254 || 0.3123 || 0.5927 || 0.3925
+
| 0.1644 || 0.1395 || 0.4002
 
|-
 
|-
 
! MMG3
 
! MMG3
−
| 0.3298 || 0.1775 || 0.4106 || 0.2742 || 0.3853 || 0.1388 || 0.4463 || 0.2024 || 0.5254 || 0.3123 || 0.5927 || 0.3925
+
| 0.4038 || 0.2929 || 0.4556
 
         |}
 
         |}

Latest revision as of 07:55, 30 October 2019

Introduction

These are the results for the 2018 running of the Music and/or Speech Detection tasks. For background information about this task set please refer to the 2018:Music and/or Speech Detection page.

General Legend

Sub code Abstract Contributors
MMG1 PDF Blai Meléndez-Catalán, Emilio Molina, Emilia Gómez
MMG2 PDF Blai Meléndez-Catalán, Emilio Molina, Emilia Gómez
MMG3 PDF Blai Meléndez-Catalán, Emilio Molina, Emilia Gómez

Statistics notation

Accuracy = segment-level accuracy

<class>_P = segment-level precision for the class <class>

<class>_R = segment-level recall for the class <class>

<class>_F = segment-level F-measure for the class <class>

<class>_F_500_on = onset-only event-level F-measure (500 ms tolerance) for the class <class>

<class>_F_500_onoff = onset-offset event-level F-measure (500 ms tolerance) for the class <class>

<class>_F_1000_on = onset-only event-level F-measure (1000 ms tolerance) for the class <class>

<class>_F_1000_onoff = onset-offset event-level F-measure (1000 ms tolerance) for the class <class>

Datasets description

Dataset description

Task 1: Music Detection

Segment-level Evaluation

Sub code Accuracy Music_P Music_R Music_F No-Music_P No-Music_R No-Music_F
MMG1 0.8713 0.9056 0.8513 0.8776 0.8354 0.8953 0.8643
MMG2 0.8928 0.9186 0.8803 0.899 0.8648 0.9079 0.8858
MMG3 0.9178 0.9026 0.9511 0.9262 0.9381 0.8787 0.9074

Event-level Evaluation

Sub code Music_F_500_on Music_F_500_onoff Music_F_1000_on Music_F_1000_onoff
MMG1 0.5176 0.2456 0.5664 0.3108
MMG2 0.5486 0.2748 0.5961 0.342
MMG3 0.6456 0.383 0.6903 0.4538

Task 2: Relative Music Loudness Estimation

Segment-level Evaluation

Sub code Accuracy Fg-Music_P Fg-Music_R Fg-Music_F Bg-Music_P Bg-Music_R Bg-Music_F No-Music_P No-Music_R No-Music_F
MMG1 0.8152 0.7874 0.7392 0.7626 0.7941 0.7647 0.7791 0.8398 0.8834 0.861
MMG2 0.8414 0.8741 0.7055 0.7808 0.8035 0.8154 0.8095 0.8648 0.9079 0.8858
MMG3 0.8749 0.8375 0.8291 0.8333 0.8238 0.8896 0.8554 0.9387 0.8773 0.907

Event-level Evaluation

Onset 500 ms tolerance
Sub code Fg-Music_F_500_on Bg-Music_F_500_on No-Music_F_500_on
MMG1 0.2413 0.3112 0.4729
MMG2 0.2418 0.3424 0.5178
MMG3 0.4852 0.4916 0.5627
Onset-offset 500 ms tolerance
Sub code Fg-Music_F_500_onoff Bg-Music_F_500_onoff No-Music_F_500_onoff
MMG1 0.1045 0.0777 0.2646
MMG2 0.1172 0.0934 0.3329
MMG3 0.3136 0.2287 0.38
Onset 1000 ms tolerance
Sub code Fg-Music_F_1000_on Bg-Music_F_1000_on No-Music_F_1000_on
MMG1 0.3078 0.3629 0.525
MMG2 0.3033 0.3932 0.5748
MMG3 0.5493 0.5414 0.6213
Onset-offset 1000 ms tolerance
Sub code Fg-Music_F_1000_onoff Bg-Music_F_1000_onoff No-Music_F_1000_onoff
MMG1 0.1584 0.1236 0.3283
MMG2 0.1644 0.1395 0.4002
MMG3 0.4038 0.2929 0.4556
Retrieved from "https://music-ir.org/mirex/w/index.php?title=2019:Music_Detection_Results&oldid=13066"