Difference between revisions of "2019:Audio Chord Estimation Results"

From MIREX Wiki
(Created page with "==Introduction== This page contains the results of the 2018 edition of the MIREX automatic chord estimation tasks. This edition was the sixth one since the reorganization of...")
 
Line 1: Line 1:
 
==Introduction==
 
==Introduction==
  
−
This page contains the results of the 2018 edition of the MIREX automatic chord estimation tasks. This edition was the sixth one since the reorganization of the evaluation procedure in 2013. The results can therefore be directly compared to those of the last five years. Chord labels are evaluated according to five different chord vocabularies and the segmentation is also assessed. Additional information about the used measures can be found on the page of the [[2013:Audio_Chord_Estimation_Results_MIREX_2009 | 2013 edition]].  
+
This page contains the results of the 2019 edition of the MIREX automatic chord estimation tasks. This edition was the sixth one since the reorganization of the evaluation procedure in 2013. The results can therefore be directly compared to those of the last five years. Chord labels are evaluated according to five different chord vocabularies and the segmentation is also assessed. Additional information about the used measures can be found on the page of the [[2013:Audio_Chord_Estimation_Results_MIREX_2009 | 2013 edition]].  
  
 
==What’s new?==
 
==What’s new?==
Line 17: Line 17:
 
! Abstract
 
! Abstract
 
! Contributors
 
! Contributors
 +
|-
 +
| CLSYJ1
 +
| style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2019/CLSYJ1.pdf PDF]
 +
| [http://mirlab.org/users/eden.chien/ i Chien], [http://mirlab.org/users/hubert.lee/ Song Rong Lee], [http://mirlab.org/ Yeh Ssuhung], [http://mirlab.org/users/kenshincs/index.htm Tzu-Chun Yeh], [http://mirlab.org/jang Jyh-Shing Roger Jang]
 
|-
 
|-
 
| CM1
 
| CM1
−
| style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2018/CM1.pdf PDF]
+
| style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2019/CM1.pdf PDF]
−
| [https://code.soundsoftware.ac.uk/users/3 Chris Cannam], [http://c4dm.eecs.qmul.ac.uk/ Matthias Mauch]
+
| [https://code.soundsoftware.ac.uk/users/3 Chris Cannam], [http://matthiasmauch.net Matthias Mauch]
−
|-
 
−
| JLCX1, JLCX2
 
−
| style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2018/JLCX1.pdf PDF]
 
−
| [https://github.com/instr3/ Junyan Jiang],  [https://github.com/RetroCirce Ke Chen], [http://www.cs.fudan.edu.cn/ Wei Li], [http://www.cs.cmu.edu/~gxia Guangyu Xia]
 
−
|-
 
−
| SG1
 
−
| style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2018/SG1.pdf PDF]
 
−
| [https://www.fsit.services Franz Strasser], [http://www.jku.at/ Stefan Gaser]  
 
 
|-
 
|-
−
| FK2
 
−
| style="text-align: center;" | [PDF]
 
−
| [http://www.cp.jku.at Florian Krebs], [http://www.cp.jku.at Filip Korzeniowski], [http://www.ofai.at Sebastian Böck]
 
 
|}
 
|}
  

Revision as of 12:27, 4 November 2019

Introduction

This page contains the results of the 2019 edition of the MIREX automatic chord estimation tasks. This edition was the sixth one since the reorganization of the evaluation procedure in 2013. The results can therefore be directly compared to those of the last five years. Chord labels are evaluated according to five different chord vocabularies and the segmentation is also assessed. Additional information about the used measures can be found on the page of the 2013 edition.

What’s new?

  • All datasets and evaluation procedures are the same as last year's MIREX.

Software

All software used for the evaluation has been made open-source. The evaluation framework is described by Pauwels and Peeters (2013). The corresponding binaries and code repository can be found on GitHub and the used measures are available as presets. The raw algorithmic output will be provided later More help can be found in the readme.

The statistical comparison between the different submissions is explained in Burgoyne et al. (2014). The software is available at BitBucket. It uses the detailed results provided below as input.

Submissions

Abstract Contributors
CLSYJ1 PDF i Chien, Song Rong Lee, Yeh Ssuhung, Tzu-Chun Yeh, Jyh-Shing Roger Jang
CM1 PDF Chris Cannam, Matthias Mauch

Results

Summary

All figures can be interpreted as percentages and range from 0 (worst) to 100 (best).

Isophonics2009
Algorithm MirexRoot MirexMajMin MirexMajMinBass MirexSevenths MirexSeventhsBass MeanSeg UnderSeg OverSeg
CLSYJ1 79.89 76.50 74.33 67.79 65.82 82.61 81.56 85.11
CM1 78.66 75.51 72.58 54.78 52.36 85.87 87.22 85.98

download these results as csv

Billboard2012
Algorithm MirexRoot MirexMajMin MirexMajMinBass MirexSevenths MirexSeventhsBass MeanSeg UnderSeg OverSeg
CLSYJ1 76.32 73.38 71.60 63.59 61.92 82.05 81.74 83.77
CM1 74.23 72.31 70.25 55.44 53.48 83.60 85.33 83.31

download these results as csv

Billboard2013
Algorithm MirexRoot MirexMajMin MirexMajMinBass MirexSevenths MirexSeventhsBass MeanSeg UnderSeg OverSeg
CLSYJ1 73.69 69.47 67.66 57.41 55.78 78.80 78.67 82.52
CM1 71.23 67.36 65.28 49.07 47.25 81.50 83.13 82.53

download these results as csv

JayChou29
Algorithm MirexRoot MirexMajMin MirexMajMinBass MirexSevenths MirexSeventhsBass MeanSeg UnderSeg OverSeg
CLSYJ1 65.44 62.49 55.27 46.83 40.60 78.95 80.78 77.55
CM1 72.82 72.15 65.55 54.46 49.05 86.58 86.94 86.84

download these results as csv

RobbieWilliams
Algorithm MirexRoot MirexMajMin MirexMajMinBass MirexSevenths MirexSeventhsBass MeanSeg UnderSeg OverSeg
CLSYJ1 79.89 75.45 73.73 68.57 66.91 83.60 83.72 84.32
CM1 81.94 78.29 76.09 57.97 55.94 87.95 89.00 87.39

download these results as csv

RWC-Popular
Algorithm MirexRoot MirexMajMin MirexMajMinBass MirexSevenths MirexSeventhsBass MeanSeg UnderSeg OverSeg
CLSYJ1 70.33 66.77 63.42 56.09 52.97 79.26 79.17 80.14
CM1 79.12 77.95 74.30 63.33 59.89 88.62 88.11 89.67

download these results as csv

USPOP2002Chords
Algorithm MirexRoot MirexMajMin MirexMajMinBass MirexSevenths MirexSeventhsBass MeanSeg UnderSeg OverSeg
CLSYJ1 76.23 72.30 68.75 62.27 58.92 82.13 82.12 83.24
CM1 78.41 76.24 72.74 59.84 56.67 85.81 86.81 86.02

download these results as csv


Detailed Results

More details about the performance of the algorithms, including per-song performance and supplementary statistics, are available from ?.

Algorithmic Output

The raw output of the algorithms are available in ?. They can be used to experiment with alternative evaluation measures and statistics.

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2019:Audio_Chord_Estimation_Results&oldid=13113"