Difference between revisions of "2025:Audio Chord Estimation Results"

From MIREX Wiki
(Created page with "This page is still WIP. More submissions and descriptions may appear. == Billboard2012 == {| class="wikitable" style="text-align:right; |- style="font-weight:bold; text-alig...")
 
Line 1: Line 1:
 
This page is still WIP. More submissions and descriptions may appear.
 
This page is still WIP. More submissions and descriptions may appear.
  
−
== Billboard2012 ==
+
= Submissions =
  
−
{| class="wikitable" style="text-align:right;
+
{| class="wikitable"  
−
|- style="font-weight:bold; text-align:left;"
+
|-
−
! style="vertical-align:bottom;" | Group
+
! Submission
−
! style="vertical-align:bottom;" | MirexRoot
+
! Title
−
! style="vertical-align:bottom;" | MirexMajMin
+
! PDF
−
! style="vertical-align:bottom;" | MirexMajMinBass
+
|-
−
! style="vertical-align:bottom;" | MirexSevenths
+
| style="vertical-align:bottom; background-color:#F8F9FA; color:#222;" | Baseline: Chordino
−
! style="vertical-align:bottom;" | MirexSeventhsBass
+
| style="vertical-align:bottom; background-color:#F8F9FA; color:#222;" | NNLS Chroma v1.1
−
! style="vertical-align:bottom;" | MeanSeg
+
| [https://github.com/ismir-mirex/ace-task-captain-notes?tab=readme-ov-file#baselines Link]
−
! style="vertical-align:bottom;" | UnderSeg
+
|-
−
! style="vertical-align:bottom;" | OverSeg
+
| Baseline: ISMIR2019
−
|- style="vertical-align:bottom;"
+
| Large-Vocabulary Chord Transcription via Chord Structure Decomposition
−
| style="text-align:left;" | Baseline: Chordino
+
| [https://github.com/ismir-mirex/ace-task-captain-notes?tab=readme-ov-file#baselines Link]
−
| 74.04
+
|-
−
| 72.11
+
| MD1
−
| 70.05
+
| Degree-Based Automatic Chord Recognition with Enharmonic Distinction
−
| 55.24
+
| TBA
−
| 53.28
+
|-
−
| 0.84
+
| wu-ensemble
−
| 0.85
+
| wu-ensemble
−
| 0.83
+
| TBA
−
|- style="vertical-align:bottom;"
+
|-
−
| style="text-align:left;" | MD1
+
| wu-single
−
| 85.11
+
| wu-single
−
| 83.98
+
| TBA
−
| 82.76
+
|-
−
| 74.12
+
| YK1
−
| 73.12
+
| Semi-Supervised Audio Chord Estimator Based on Disentangled Generative Modeling
−
| 0.89
+
| TBA
−
| 0.89
+
|-
−
| 0.90
+
| BMACE
−
|- style="vertical-align:bottom;"
+
| A Mamba-Based Model for Automatic Chord Recognition
−
| style="text-align:left;" | wu-ensemble
+
| TBA
−
| 78.26
 
−
| 77.15
 
−
| 75.58
 
−
| 59.99
 
−
| 58.79
 
−
| 0.84
 
−
| 0.88
 
−
| 0.83
 
−
|- style="vertical-align:bottom;"
 
−
| style="text-align:left;" | wu-single
 
−
| 79.23
 
−
| 78.21
 
−
| 76.76
 
−
| 60.23
 
−
| 59.07
 
−
| 0.85
 
−
| 0.87
 
−
| 0.84
 
−
|- style="vertical-align:bottom;"
 
−
| style="text-align:left;" | YK1
 
−
| 85.90
 
−
| 84.66
 
−
| 81.81
 
−
| 77.22
 
−
| 74.45
 
−
| 0.88
 
−
| 0.88
 
−
| 0.90
 
 
|}
 
|}
 +
 +
= Main Evaluation Results =
 +
 +
The following datasets are served as pure test sets. No system is allowed to train on them.
  
 
== Billboard2013 ==
 
== Billboard2013 ==
Line 139: Line 115:
 
| 0.85
 
| 0.85
 
| 0.87
 
| 0.87
−
|}
 
−
 
−
== RWC-Popular ==
 
−
 
−
{| class="wikitable" style="text-align:right;
 
−
|- style="font-weight:bold; text-align:left;"
 
−
! style="vertical-align:bottom;" | Group
 
−
! style="vertical-align:bottom;" | MirexRoot
 
−
! style="vertical-align:bottom;" | MirexMajMin
 
−
! style="vertical-align:bottom;" | MirexMajMinBass
 
−
! style="vertical-align:bottom;" | MirexSevenths
 
−
! style="vertical-align:bottom;" | MirexSeventhsBass
 
−
! style="vertical-align:bottom;" | MeanSeg
 
−
! style="vertical-align:bottom;" | UnderSeg
 
−
! style="vertical-align:bottom;" | OverSeg
 
−
|- style="vertical-align:bottom;"
 
−
| style="text-align:left;" | Baseline: Chordino
 
−
| 78.97
 
−
| 77.78
 
−
| 74.13
 
−
| 63.15
 
−
| 59.72
 
−
| 0.89
 
−
| 0.88
 
−
| 0.90
 
−
|- style="vertical-align:bottom;"
 
−
| style="text-align:left;" | MD1
 
−
| 83.98
 
−
| 81.18
 
−
| 79.42
 
−
| 66.53
 
−
| 64.83
 
−
| 0.89
 
−
| 0.89
 
−
| 0.89
 
−
|- style="vertical-align:bottom;"
 
−
| style="text-align:left;" | wu-ensemble
 
−
| 81.87
 
−
| 80.30
 
−
| 77.58
 
−
| 62.65
 
−
| 60.25
 
−
| 0.88
 
−
| 0.90
 
−
| 0.86
 
−
|- style="vertical-align:bottom;"
 
−
| style="text-align:left;" | wu-single
 
−
| 82.48
 
−
| 81.35
 
−
| 78.48
 
−
| 62.86
 
−
| 60.28
 
−
| 0.88
 
−
| 0.89
 
−
| 0.87
 
−
|- style="vertical-align:bottom;"
 
−
| style="text-align:left;" | YK1
 
−
| 88.76
 
−
| 87.27
 
−
| 81.14
 
−
| 76.88
 
−
| 70.90
 
−
| 0.92
 
−
| 0.92
 
−
| 0.92
 
 
|}
 
|}
  
Line 354: Line 265:
 
| 0.90
 
| 0.90
 
| 0.89
 
| 0.89
 +
|}
 +
 +
= Additional Test Sets =
 +
 +
Below are datasets that are not pure test sets. Some models might have trained on them; please see their extended abstracts for details.
 +
 +
== Billboard2012 ==
 +
 +
{| class="wikitable" style="text-align:right;
 +
|- style="font-weight:bold; text-align:left;"
 +
! style="vertical-align:bottom;" | Group
 +
! style="vertical-align:bottom;" | MirexRoot
 +
! style="vertical-align:bottom;" | MirexMajMin
 +
! style="vertical-align:bottom;" | MirexMajMinBass
 +
! style="vertical-align:bottom;" | MirexSevenths
 +
! style="vertical-align:bottom;" | MirexSeventhsBass
 +
! style="vertical-align:bottom;" | MeanSeg
 +
! style="vertical-align:bottom;" | UnderSeg
 +
! style="vertical-align:bottom;" | OverSeg
 +
|- style="vertical-align:bottom;"
 +
| style="text-align:left;" | Baseline: Chordino
 +
| 74.04
 +
| 72.11
 +
| 70.05
 +
| 55.24
 +
| 53.28
 +
| 0.84
 +
| 0.85
 +
| 0.83
 +
|- style="vertical-align:bottom;"
 +
| style="text-align:left;" | MD1
 +
| 85.11
 +
| 83.98
 +
| 82.76
 +
| 74.12
 +
| 73.12
 +
| 0.89
 +
| 0.89
 +
| 0.90
 +
|- style="vertical-align:bottom;"
 +
| style="text-align:left;" | wu-ensemble
 +
| 78.26
 +
| 77.15
 +
| 75.58
 +
| 59.99
 +
| 58.79
 +
| 0.84
 +
| 0.88
 +
| 0.83
 +
|- style="vertical-align:bottom;"
 +
| style="text-align:left;" | wu-single
 +
| 79.23
 +
| 78.21
 +
| 76.76
 +
| 60.23
 +
| 59.07
 +
| 0.85
 +
| 0.87
 +
| 0.84
 +
|- style="vertical-align:bottom;"
 +
| style="text-align:left;" | YK1
 +
| 85.90
 +
| 84.66
 +
| 81.81
 +
| 77.22
 +
| 74.45
 +
| 0.88
 +
| 0.88
 +
| 0.90
 +
|}
 +
 +
 +
== RWC-Popular ==
 +
 +
{| class="wikitable" style="text-align:right;
 +
|- style="font-weight:bold; text-align:left;"
 +
! style="vertical-align:bottom;" | Group
 +
! style="vertical-align:bottom;" | MirexRoot
 +
! style="vertical-align:bottom;" | MirexMajMin
 +
! style="vertical-align:bottom;" | MirexMajMinBass
 +
! style="vertical-align:bottom;" | MirexSevenths
 +
! style="vertical-align:bottom;" | MirexSeventhsBass
 +
! style="vertical-align:bottom;" | MeanSeg
 +
! style="vertical-align:bottom;" | UnderSeg
 +
! style="vertical-align:bottom;" | OverSeg
 +
|- style="vertical-align:bottom;"
 +
| style="text-align:left;" | Baseline: Chordino
 +
| 78.97
 +
| 77.78
 +
| 74.13
 +
| 63.15
 +
| 59.72
 +
| 0.89
 +
| 0.88
 +
| 0.90
 +
|- style="vertical-align:bottom;"
 +
| style="text-align:left;" | MD1
 +
| 83.98
 +
| 81.18
 +
| 79.42
 +
| 66.53
 +
| 64.83
 +
| 0.89
 +
| 0.89
 +
| 0.89
 +
|- style="vertical-align:bottom;"
 +
| style="text-align:left;" | wu-ensemble
 +
| 81.87
 +
| 80.30
 +
| 77.58
 +
| 62.65
 +
| 60.25
 +
| 0.88
 +
| 0.90
 +
| 0.86
 +
|- style="vertical-align:bottom;"
 +
| style="text-align:left;" | wu-single
 +
| 82.48
 +
| 81.35
 +
| 78.48
 +
| 62.86
 +
| 60.28
 +
| 0.88
 +
| 0.89
 +
| 0.87
 +
|- style="vertical-align:bottom;"
 +
| style="text-align:left;" | YK1
 +
| 88.76
 +
| 87.27
 +
| 81.14
 +
| 76.88
 +
| 70.90
 +
| 0.92
 +
| 0.92
 +
| 0.92
 
|}
 
|}

Revision as of 23:34, 7 September 2025

This page is still WIP. More submissions and descriptions may appear.

Submissions

Submission Title PDF
Baseline: Chordino NNLS Chroma v1.1 Link
Baseline: ISMIR2019 Large-Vocabulary Chord Transcription via Chord Structure Decomposition Link
MD1 Degree-Based Automatic Chord Recognition with Enharmonic Distinction TBA
wu-ensemble wu-ensemble TBA
wu-single wu-single TBA
YK1 Semi-Supervised Audio Chord Estimator Based on Disentangled Generative Modeling TBA
BMACE A Mamba-Based Model for Automatic Chord Recognition TBA

Main Evaluation Results

The following datasets are served as pure test sets. No system is allowed to train on them.

Billboard2013

Group MirexRoot MirexMajMin MirexMajMinBass MirexSevenths MirexSeventhsBass MeanSeg UnderSeg OverSeg
Baseline: Chordino 71.06 67.18 65.09 48.88 47.06 0.82 0.83 0.83
Baseline: ISMIR2019 78.61 76.39 74.72 64.15 62.65 0.83 0.79 0.93
MD1 81.35 79.15 77.91 66.40 65.33 0.86 0.85 0.89
wu-ensemble 74.64 71.97 70.72 55.06 53.96 0.83 0.86 0.82
wu-single 75.77 73.14 71.74 55.41 54.15 0.83 0.85 0.83
YK1 81.01 78.10 75.41 64.53 62.05 0.86 0.85 0.87

YAMAHA_Balanced

Group MirexRoot MirexMajMin MirexMajMinBass MirexSevenths MirexSeventhsBass MeanSeg UnderSeg OverSeg
Baseline: Chordino 77.57 74.64 71.59 56.38 53.90 0.87 0.87 0.87
Baseline: ISMIR2019 82.00 81.16 79.69 66.97 65.77 0.89 0.86 0.93
MD1 81.83 80.22 78.87 64.13 63.14 0.88 0.89 0.88
wu-ensemble 82.54 81.29 78.99 62.84 60.84 0.87 0.89 0.87
wu-single 81.37 79.69 77.61 61.60 59.84 0.87 0.90 0.86
YK1 82.53 79.71 75.60 66.02 62.31 0.89 0.90 0.89

YAMAHA_JPop

Group MirexRoot MirexMajMin MirexMajMinBass MirexSevenths MirexSeventhsBass MeanSeg UnderSeg OverSeg
Baseline: Chordino 74.49 71.99 69.24 52.40 49.97 0.87 0.86 0.88
Baseline: ISMIR2019 81.49 79.99 78.58 62.81 61.61 0.90 0.87 0.94
MD1 79.34 77.10 76.07 55.59 54.71 0.88 0.88 0.88
wu-ensemble 79.58 77.58 75.57 54.36 52.58 0.87 0.88 0.87
wu-single 78.87 76.56 74.66 55.35 53.60 0.87 0.89 0.86
YK1 80.13 77.03 72.85 61.24 57.26 0.89 0.90 0.89

Additional Test Sets

Below are datasets that are not pure test sets. Some models might have trained on them; please see their extended abstracts for details.

Billboard2012

Group MirexRoot MirexMajMin MirexMajMinBass MirexSevenths MirexSeventhsBass MeanSeg UnderSeg OverSeg
Baseline: Chordino 74.04 72.11 70.05 55.24 53.28 0.84 0.85 0.83
MD1 85.11 83.98 82.76 74.12 73.12 0.89 0.89 0.90
wu-ensemble 78.26 77.15 75.58 59.99 58.79 0.84 0.88 0.83
wu-single 79.23 78.21 76.76 60.23 59.07 0.85 0.87 0.84
YK1 85.90 84.66 81.81 77.22 74.45 0.88 0.88 0.90


RWC-Popular

Group MirexRoot MirexMajMin MirexMajMinBass MirexSevenths MirexSeventhsBass MeanSeg UnderSeg OverSeg
Baseline: Chordino 78.97 77.78 74.13 63.15 59.72 0.89 0.88 0.90
MD1 83.98 81.18 79.42 66.53 64.83 0.89 0.89 0.89
wu-ensemble 81.87 80.30 77.58 62.65 60.25 0.88 0.90 0.86
wu-single 82.48 81.35 78.48 62.86 60.28 0.88 0.89 0.87
YK1 88.76 87.27 81.14 76.88 70.90 0.92 0.92 0.92
Retrieved from "https://music-ir.org/mirex/w/index.php?title=2025:Audio_Chord_Estimation_Results&oldid=14751"