Difference between revisions of "2015:Music/Speech Classification and Detection Results"

From MIREX Wiki
(→‎General Legend)
(→‎Introduction)
 
(47 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
==Introduction==
 
==Introduction==
−
These are the results for the 2015 running of the Music/Speech Classification and Detection task. For background information about this task set please refer to the [[2015:Music/speech classification and detection]] page.
+
These are the results for the 2015 running of the Music/Speech Classification and Detection task. For background information about this task set please refer to the [[2015:Music/Speech Classification and Detection]] page.
  
 
==General Legend==
 
==General Legend==
Line 31: Line 31:
 
         |-
 
         |-
 
! RS2
 
! RS2
−
| PDF || Reinhard Sonnleitner
+
| [https://www.music-ir.org/mirex/abstracts/2015/RS2.pdf PDF]  || Reinhard Sonnleitner
 
         |-
 
         |-
 
! TL1
 
! TL1
Line 56: Line 56:
 
! ZY1
 
! ZY1
 
| [https://www.music-ir.org/mirex/abstracts/2015/ZY1.pdf PDF] || Wei-Qiang Zhang, Xu-Kui Yang
 
| [https://www.music-ir.org/mirex/abstracts/2015/ZY1.pdf PDF] || Wei-Qiang Zhang, Xu-Kui Yang
 +
        |-
 +
! TVDP1
 +
| [https://www.music-ir.org/mirex/abstracts/2015/TVDP1.pdf PDF] || Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
 +
        |-
 +
! TVDP2
 +
| [https://www.music-ir.org/mirex/abstracts/2015/TVDP2.pdf PDF] || Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
 +
        |-
 +
! TVDP3
 +
| [https://www.music-ir.org/mirex/abstracts/2015/TVDP3.pdf PDF] || Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
 +
        |-
 +
! NT1
 +
| [https://www.music-ir.org/mirex/abstracts/2015/NT1.pdf PDF] || Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
 +
        |-
 +
! NT2
 +
| [https://www.music-ir.org/mirex/abstracts/2015/NT2.pdf PDF] || Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
 +
        |-
 +
! YZ1
 +
| [https://www.music-ir.org/mirex/abstracts/2015/YZ1.pdf PDF] || Wei-Qiang Zhang, Xu-Kui Yang
 +
        |}
 +
 +
==Task 1: Music/Speech Classification==
 +
 +
'''Complete dataset'''
 +
 +
{| border="1" cellspacing="0" style="text-align: left; width: 240px;"
 +
|- style="background: yellow;"
 +
! width="80" | Sub code
 +
! width="80" style="text-align: center;" | Accuracy
 +
! width="80" | std
 +
|-
 +
! GWDS1
 +
| 0.9866 || 0.0065
 +
        |-
 +
! JS3
 +
| 0.9946 || 0.0027
 +
        |-
 +
! MM1
 +
| 0.9869 || 0.0037
 +
        |-
 +
! MM2
 +
| 0.9754 || 0.007
 +
        |-
 +
! RS2
 +
| 0.9962 || 0.0029
 +
        |-
 +
! TL1(15)
 +
| 0.9927 || 0.0018
 +
        |-
 +
! TL1(80)
 +
| 0.9973 || 0.0018
 +
        |-
 +
! ZC4
 +
| 0.9429 || 0.0076
 +
        |-
 +
! ZC5
 +
| 0.9429 || 0.0076
 +
        |-
 +
! NT1
 +
| 0.9835 || 0.0109
 +
        |-
 +
! NT2
 +
| 0.9935 || 0.0044
 +
        |-
 +
! RHM1
 +
| 0.9954 || 0.004
 +
        |-
 +
! YZ1
 +
| 0.9835 || 0.0044
 +
        |}
 +
 +
===Individual Results Files for Task 1===
 +
 +
'''GWDS1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/GWDS1_confusion_matrix.csv Vikaskumar Ghodasara, Daimi Syed Naser, Shefali Waldekar, Goutam Saha ]<br />
 +
'''JS3'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/JS3_confusion_matrix.csv Jan Schlüter ]<br />
 +
'''MM1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/MM1_confusion_matrix.csv Matija Marolt ]<br />
 +
'''MM2'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/MM2_confusion_matrix.csv Matija Marolt ]<br />
 +
'''RS2'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/RS2_confusion_matrix.csv Reinhard Sonnleitner ]<br />
 +
'''TL1(15)'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/TL1_confusion_matrix.csv Thomas Lidy ]<br />
 +
'''TL1(80)'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/TL1-80_confusion_matrix.csv Thomas Lidy ]<br />
 +
'''ZC4'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/ZC4_confusion_matrix.csv Chaogang Zhang, Chuan-Yi Chen ]<br />
 +
'''ZC5'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/ZC5_confusion_matrix.csv Chaogang Zhang, Chuan-Yi Chen ]<br />
 +
'''NT1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/NT1_confusion_matrix.csv Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou ]<br />
 +
'''NT2'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/NT2_confusion_matrix.csv Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou ]<br />
 +
'''RHM1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/RHM1_confusion_matrix.csv Jimena Royo-Letelier, Romain Hennequin, Manuel Moussallam ]<br />
 +
'''YZ1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-classification-confusion-matrices/YZ1_confusion_matrix.csv Wei-Qiang Zhang, Xu-Kui Yang ]<br />
 +
 +
==Task 2: Music/Speech Detection==
 +
 +
{| border="1" cellspacing="0" style="text-align: left; width: 600px;"
 +
|- style="background: yellow;"
 +
! width="80" | Sub code
 +
! width="80" | fb_F
 +
! width="80" | eb_F_500ms
 +
! width="80" | eb_Foff_500ms
 +
! width="80" | eb_F_1s
 +
! width="80" | eb_Foff_1s
 +
|-
 +
! JS2
 +
| 0.8233 || 0.1754 || 0.1212 || 0.2387 || 0.1639
 +
|-
 +
! MM3
 +
| 0.8941 || 0.4037 || 0.2911 || 0.4419 || 0.3085
 +
|-
 +
! TVDP1
 +
| 0.8314 || 0.1490 || 0.1146 || 0.2233 || 0.1718
 +
|-
 +
! TVDP2
 +
| 0.8346 || 0.1250 || 0.1231 || 0.2042 || 0.1741
 +
|-
 +
! TVDP3
 +
| 0.8209 || 0.0699 || 0.0068 || 0.1299 || 0.0306
 +
|-
 +
! UK1
 +
| 0.5483 || 0.0324 || 0.0037 || 0.0623 || 0.0070
 +
|-
 +
! ZC1
 +
| 0.7517 || 0.2168 || 0.1956 || 0.2700 || 0.2101
 +
|-
 +
! ZC2
 +
| 0.2079 || 0.0434 || 0.0434 || 0.0434 || 0.0434
 +
|-
 +
! ZC3
 +
| 0.7517 || 0.2168 || 0.1956 || 0.2700 || 0.2101
 +
|-
 +
! ZC4
 +
| 0.7517 || 0.2168 || 0.1956 || 0.2700 || 0.2101
 +
|-
 +
! ZC5
 +
| 0.7517 || 0.2168 || 0.1956 || 0.2700 || 0.2101
 +
|-
 +
! ZY1
 +
| 0.5104 || 0.1670 || 0.0874 || 0.2164 || 0.1177
 +
|-
 +
! TL1
 +
| 0.8849 || 0.2129 || 0.1129 || 0.2556 || 0.1498
 
         |}
 
         |}
 +
 +
'''Notes on metrics:'''
 +
 +
fb_F=frame-based F-measure
 +
 +
eb_F_500ms=onset-only event-based F-measure (500ms tolerance)
 +
 +
eb_Foff_500ms=onset-offset event-based F-measure (500ms tolerance)
 +
 +
eb_F_1s=onset-only event-based F-measure (1s tolerance)
 +
 +
eb_Foff_1s=onset-offset event-based F-measure (1s tolerance)
 +
 +
 +
===Individual Results Files for Task 2===
 +
 +
'''JS2'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_JS2.mat Jan Schlüter ]<br />
 +
'''MM3'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_MM3.mat Matija Marolt ]<br />
 +
'''TVDP1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_TVDP1.mat Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou ]<br />
 +
'''TVDP2'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_TVDP2.mat Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou ]<br />
 +
'''TVDP3'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_TVDP3.mat Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou ]<br />
 +
'''UK1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_UK1.mat Aiko Uemura, Jiro Katto ]<br />
 +
'''ZC1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_ZC1.mat Chaogang Zhang, Chuan-Yi Chen ]<br />
 +
'''ZC2'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_ZC2.mat Chaogang Zhang, Chuan-Yi Chen ]<br />
 +
'''ZC3'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_ZC3.mat Chaogang Zhang, Chuan-Yi Chen ]<br />
 +
'''ZC4'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_ZC4.mat Chaogang Zhang, Chuan-Yi Chen ]<br />
 +
'''ZC5'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_ZC5.mat Chaogang Zhang, Chuan-Yi Chen ]<br />
 +
'''ZY1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_ZY1.mat Wei-Qiang Zhang, Xu-Kui Yang ]<br />
 +
'''TL1'''= [https://www.music-ir.org/mirex/results/2015/mscd/muspeak-detection-results/detection_results_TL1-80.mat Thomas Lidy ] (TBA) <br />
 +
 +
Note: detailed results are stored as binary Matlab files (.mat)

Latest revision as of 07:20, 25 February 2016

Introduction

These are the results for the 2015 running of the Music/Speech Classification and Detection task. For background information about this task set please refer to the 2015:Music/Speech Classification and Detection page.

General Legend

Sub code Abstract Contributors
GWDS1 PDF Vikaskumar Ghodasara, Daimi Syed Naser, Shefali Waldekar, Goutam Saha
JS2 PDF Jan Schlüter
JS3 PDF Jan Schlüter
MM1 PDF Matija Marolt
MM2 PDF Matija Marolt
MM3 PDF Matija Marolt
RHM1 PDF Jimena Royo-Letelier, Romain Hennequin, Manuel Moussallam
RS2 PDF Reinhard Sonnleitner
TL1 PDF Thomas Lidy
UK1 PDF Aiko Uemura, Jiro Katto
ZC1 PDF Chaogang Zhang, Chuan-Yi Chen
ZC2 PDF Chaogang Zhang, Chuan-Yi Chen
ZC3 PDF Chaogang Zhang, Chuan-Yi Chen
ZC4 PDF Chaogang Zhang, Chuan-Yi Chen
ZC5 PDF Chaogang Zhang, Chuan-Yi Chen
ZY1 PDF Wei-Qiang Zhang, Xu-Kui Yang
TVDP1 PDF Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
TVDP2 PDF Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
TVDP3 PDF Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
NT1 PDF Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
NT2 PDF Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
YZ1 PDF Wei-Qiang Zhang, Xu-Kui Yang

Task 1: Music/Speech Classification

Complete dataset

Sub code Accuracy std
GWDS1 0.9866 0.0065
JS3 0.9946 0.0027
MM1 0.9869 0.0037
MM2 0.9754 0.007
RS2 0.9962 0.0029
TL1(15) 0.9927 0.0018
TL1(80) 0.9973 0.0018
ZC4 0.9429 0.0076
ZC5 0.9429 0.0076
NT1 0.9835 0.0109
NT2 0.9935 0.0044
RHM1 0.9954 0.004
YZ1 0.9835 0.0044

Individual Results Files for Task 1

GWDS1= Vikaskumar Ghodasara, Daimi Syed Naser, Shefali Waldekar, Goutam Saha
JS3= Jan Schlüter
MM1= Matija Marolt
MM2= Matija Marolt
RS2= Reinhard Sonnleitner
TL1(15)= Thomas Lidy
TL1(80)= Thomas Lidy
ZC4= Chaogang Zhang, Chuan-Yi Chen
ZC5= Chaogang Zhang, Chuan-Yi Chen
NT1= Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
NT2= Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
RHM1= Jimena Royo-Letelier, Romain Hennequin, Manuel Moussallam
YZ1= Wei-Qiang Zhang, Xu-Kui Yang

Task 2: Music/Speech Detection

Sub code fb_F eb_F_500ms eb_Foff_500ms eb_F_1s eb_Foff_1s
JS2 0.8233 0.1754 0.1212 0.2387 0.1639
MM3 0.8941 0.4037 0.2911 0.4419 0.3085
TVDP1 0.8314 0.1490 0.1146 0.2233 0.1718
TVDP2 0.8346 0.1250 0.1231 0.2042 0.1741
TVDP3 0.8209 0.0699 0.0068 0.1299 0.0306
UK1 0.5483 0.0324 0.0037 0.0623 0.0070
ZC1 0.7517 0.2168 0.1956 0.2700 0.2101
ZC2 0.2079 0.0434 0.0434 0.0434 0.0434
ZC3 0.7517 0.2168 0.1956 0.2700 0.2101
ZC4 0.7517 0.2168 0.1956 0.2700 0.2101
ZC5 0.7517 0.2168 0.1956 0.2700 0.2101
ZY1 0.5104 0.1670 0.0874 0.2164 0.1177
TL1 0.8849 0.2129 0.1129 0.2556 0.1498

Notes on metrics:

fb_F=frame-based F-measure

eb_F_500ms=onset-only event-based F-measure (500ms tolerance)

eb_Foff_500ms=onset-offset event-based F-measure (500ms tolerance)

eb_F_1s=onset-only event-based F-measure (1s tolerance)

eb_Foff_1s=onset-offset event-based F-measure (1s tolerance)


Individual Results Files for Task 2

JS2= Jan Schlüter
MM3= Matija Marolt
TVDP1= Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
TVDP2= Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
TVDP3= Nikolaos Tsipas, Lazaros Vrysis, Charalampos Dimoulas, George Papanikolaou
UK1= Aiko Uemura, Jiro Katto
ZC1= Chaogang Zhang, Chuan-Yi Chen
ZC2= Chaogang Zhang, Chuan-Yi Chen
ZC3= Chaogang Zhang, Chuan-Yi Chen
ZC4= Chaogang Zhang, Chuan-Yi Chen
ZC5= Chaogang Zhang, Chuan-Yi Chen
ZY1= Wei-Qiang Zhang, Xu-Kui Yang
TL1= Thomas Lidy (TBA)

Note: detailed results are stored as binary Matlab files (.mat)

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2015:Music/Speech_Classification_and_Detection_Results&oldid=11671"