Difference between revisions of "2009:Audio Chord Detection Results"
From MIREX Wiki
JohanPauwels (talk | contribs) (→Task 1 Results Per Artist) |
IMIRSELBot (talk | contribs) m (Robot: Automated text replacement (-mirex/abs/ +mirex/abstracts/)) |
||
| (8 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
==Introduction== | ==Introduction== | ||
| − | These are the results for the 2009 running of the Audio Chord Detection task set. For background information about this task set please refer to the [[Audio Chord Detection]] page. | + | These are the results for the 2009 running of the Audio Chord Detection task set. For background information about this task set please refer to the [[2009:Audio Chord Detection]] page. |
===Task Descriptions=== | ===Task Descriptions=== | ||
| Line 18: | Line 18: | ||
====Team ID==== | ====Team ID==== | ||
| − | '''CH''' = [https://music-ir.org/mirex/2009 | + | '''CH''' = [https://www.music-ir.org/mirex/abstracts/2009/harte_mirex09.pdf Christopher Harte]<br /> |
| − | '''DE''' = [https://music-ir.org/mirex/2009 | + | '''DE''' = [https://www.music-ir.org/mirex/abstracts/2009/DE.pdf Dan Ellis]<br /> |
| − | '''KO1''' = [https://music-ir.org/mirex/2009 | + | '''KO1''' = [https://www.music-ir.org/mirex/abstracts/2009/KO.pdf Maksim Khadkevich, Maurizio Omologo]<br /> |
| − | '''KO2''' = [https://music-ir.org/mirex/2009 | + | '''KO2''' = [https://www.music-ir.org/mirex/abstracts/2009/KO.pdf Maksim Khadkevich, Maurizio Omologo]<br /> |
| − | '''MD''' = [https://music-ir.org/mirex/2009 | + | '''MD''' = [https://www.music-ir.org/mirex/abstracts/2009/ACD_SS_mauch.pdf Matthias Mauch, Katy Noland, Simon Dixon]<br /> |
| − | '''OGF1''' = [https://music-ir.org/mirex/2009 | + | '''OGF1''' = [https://www.music-ir.org/mirex/abstracts/2009/OGF1.pdf Laurent Oudre, Yves Grenier,Cedric Fevotte (majmin)]<br /> |
| − | '''OGF2''' = [https://music-ir.org/mirex/2009 | + | '''OGF2''' = [https://www.music-ir.org/mirex/abstracts/2009/OGF2.pdf Laurent Oudre, Yves Grenier,Cedric Fevotte (majmin7)]<br /> |
| − | '''PP''' = [https://music-ir.org/mirex/2009 | + | '''PP''' = [https://www.music-ir.org/mirex/abstracts/2009/PPupdated.pdf Helene Papadopoulos, Geoffroy Peeters]<br /> |
| − | '''PVM1''' = [https://music-ir.org/mirex/2009 | + | '''PVM1''' = [https://www.music-ir.org/mirex/abstracts/2009/PVM.pdf Johan Pauwels, Matthias Varewyck,Jean-Pierre Martens]<br /> |
| − | '''PVM2''' = [https://music-ir.org/mirex/2009 | + | '''PVM2''' = [https://www.music-ir.org/mirex/abstracts/2009/PVM.pdf Johan Pauwels, Matthias Varewyck,Jean-Pierre Martens]<br /> |
| − | '''RRHS1''' = [https://music-ir.org/mirex/2009 | + | '''RRHS1''' = [https://www.music-ir.org/mirex/abstracts/2009/RRHS.pdf Thomas Rocher, Matthias Robine, Pierre Hanna,Robert Strandh]<br /> |
| − | '''RSHS2''' = [https://music-ir.org/mirex/2009 | + | '''RSHS2''' = [https://www.music-ir.org/mirex/abstracts/2009/RRHS.pdf Thomas Rocher, Matthias Robine, Pierre Hanna,Robert Strandh]<br /> |
| − | '''RSHS3''' = [https://music-ir.org/mirex/2009 | + | '''RSHS3''' = [https://www.music-ir.org/mirex/abstracts/2009/RRHS.pdf Thomas Rocher, Matthias Robine, Pierre Hanna,Robert Strandh]<br /> |
| − | '''RUSUSL''' = [https://music-ir.org/mirex/2009 | + | '''RUSUSL''' = [https://www.music-ir.org/mirex/abstracts/2009/RUSUS.pdf J.T.Reed,Yushi Ueda,S.Siniscalchi,Yuki Uchiyama,Shigeki Sagayama,C.H.Lee]<br /> |
| − | '''WEJ1''' = [https://music-ir.org/mirex/2009 | + | '''WEJ1''' = [https://www.music-ir.org/mirex/abstracts/2009/WEJ.pdf Adrian Weller, Daniel Ellis, Tony Jebara]<br /> |
| − | '''WEJ2''' = [https://music-ir.org/mirex/2009 | + | '''WEJ2''' = [https://www.music-ir.org/mirex/abstracts/2009/WEJ.pdf Adrian Weller, Daniel Ellis, Tony Jebara]<br /> |
| − | '''WEJ3''' = [https://music-ir.org/mirex/2009 | + | '''WEJ3''' = [https://www.music-ir.org/mirex/abstracts/2009/WEJ.pdf Adrian Weller, Daniel Ellis, Tony Jebara]<br /> |
| − | '''WEJ4''' = [https://music-ir.org/mirex/2009 | + | '''WEJ4''' = [https://www.music-ir.org/mirex/abstracts/2009/WEJ.pdf Adrian Weller, Daniel Ellis, Tony Jebara]<br /> |
== Overall Summary Results == | == Overall Summary Results == | ||
| Line 43: | Line 43: | ||
====Task 1 Overall Results==== | ====Task 1 Overall Results==== | ||
| − | <csv>chord/task1/summary/task1.overall.csv</csv> | + | <csv>2009/chord/task1/summary/task1.overall.csv</csv> |
=====Task 1 Run Times===== | =====Task 1 Run Times===== | ||
TBA | TBA | ||
| − | <csv>chord/task1/task1_runtimes.csv</csv> | + | <csv>2009/chord/task1/task1_runtimes.csv</csv> |
| − | =====Task 1 Results Per Artist===== | + | =====Task 1 Results Per Artist/Album===== |
| − | [[Audio Chord Detection Results Per | + | [[2009:Audio Chord Detection Results Per Album#Task 1]] |
=====Task 1 Friedman's Test for Significant Differences===== | =====Task 1 Friedman's Test for Significant Differences===== | ||
| Line 57: | Line 57: | ||
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05); | Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05); | ||
| − | <csv>chord/task1/summary/Accuracy_Per_Song_Friedman_Mean_Rankstask1.friedman.Friedman_TukeyKramerHSD.csv</csv> | + | <csv>2009/chord/task1/summary/Accuracy_Per_Song_Friedman_Mean_Rankstask1.friedman.Friedman_TukeyKramerHSD.csv</csv> |
| − | https://music-ir.org/mirex/2009 | + | https://music-ir.org/mirex/results/2009/chord/task1/summary/small.Accuracy_Per_Song_Friedman_Mean_Rankstask1.friedman.Friedman_Mean_Ranks.png |
===Task 2 Results=== | ===Task 2 Results=== | ||
====Task 2 Overall Results==== | ====Task 2 Overall Results==== | ||
| − | <csv>chord/task2/summary/task2.overall.csv</csv> | + | <csv>2009/chord/task2/summary/task2.overall.csv</csv> |
=====Task 2 Run Times===== | =====Task 2 Run Times===== | ||
TBA | TBA | ||
| − | <csv>chord/task2/task2_runtimes.csv</csv> | + | <csv>2009/chord/task2/task2_runtimes.csv</csv> |
| + | |||
| + | =====Task 2 Results Per Artist/Album===== | ||
| + | [[2009:Audio Chord Detection Results Per Album#Task 2]] | ||
=====Task 2 Friedman's Test for Significant Differences===== | =====Task 2 Friedman's Test for Significant Differences===== | ||
| Line 75: | Line 78: | ||
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05); | Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05); | ||
| − | <csv>chord/task2/summary/Accuracy_Per_Song_Friedman_Mean_Rankstask2.friedman.Friedman_TukeyKramerHSD.csv</csv> | + | <csv>2009/chord/task2/summary/Accuracy_Per_Song_Friedman_Mean_Rankstask2.friedman.Friedman_TukeyKramerHSD.csv</csv> |
| − | https://music-ir.org/mirex/2009 | + | https://music-ir.org/mirex/results/2009/chord/task2/summary/small.Accuracy_Per_Song_Friedman_Mean_Rankstask2.friedman.Friedman_Mean_Ranks.png |
=== Individual Results === | === Individual Results === | ||
==== Task1 ==== | ==== Task1 ==== | ||
| − | '''CH''' = [https://music-ir.org/mirex/2009 | + | '''CH''' = [https://music-ir.org/mirex/results/2009/chord/task1/CH.task1.overlap.results.csv Christopher Harte]<br /> |
| − | '''DE''' = [https://music-ir.org/mirex/2009 | + | '''DE''' = [https://music-ir.org/mirex/results/2009/chord/task1/DE.task1.overlap.results.csv Dan Ellis]<br /> |
| − | '''KO1''' = [https://music-ir.org/mirex/2009 | + | '''KO1''' = [https://music-ir.org/mirex/results/2009/chord/task1/KO1.task1.overlap.results.csv Maksim Khadkevich, Maurizio Omologo]<br /> |
| − | '''KO2''' = [https://music-ir.org/mirex/2009 | + | '''KO2''' = [https://music-ir.org/mirex/results/2009/chord/task1/KO2.task1.overlap.results.csv Maksim Khadkevich, Maurizio Omologo]<br /> |
| − | '''MD''' = [https://music-ir.org/mirex/2009 | + | '''MD''' = [https://music-ir.org/mirex/results/2009/chord/task1/MD.task1.overlap.results.csv Matthias Mauch, Simon Dixon]<br /> |
| − | '''OGF1''' = [https://music-ir.org/mirex/2009 | + | '''OGF1''' = [https://music-ir.org/mirex/results/2009/chord/task1/OGF1.task1.overlap.results.csv Laurent Oudre,Yves Grenier,Cedric Fevotte]<br /> |
| − | '''OGF2''' = [https://music-ir.org/mirex/2009 | + | '''OGF2''' = [https://music-ir.org/mirex/results/2009/chord/task1/OGF2.task1.overlap.results.csv Laurent Oudre,Yves Grenier,Cedric Fevotte]<br /> |
| − | '''PP''' = [https://music-ir.org/mirex/2009 | + | '''PP''' = [https://music-ir.org/mirex/results/2009/chord/task1/PP.task1.overlap.results.csv Helene Papadopoulos, Geoffroy Peeters]<br /> |
| − | '''PVM1''' = [https://music-ir.org/mirex/2009 | + | '''PVM1''' = [https://music-ir.org/mirex/results/2009/chord/task1/PVM1.task1.overlap.results.csv Johan Pauwels,Matthias Varewyck,Jean-Pierre Martens]<br /> |
| − | '''PVM2''' = [https://music-ir.org/mirex/2009 | + | '''PVM2''' = [https://music-ir.org/mirex/results/2009/chord/task1/PVM2.task1.overlap.results.csv Johan Pauwels,Matthias Varewyck,Jean-Pierre Martens]<br /> |
| − | '''RRHS1''' = [https://music-ir.org/mirex/2009 | + | '''RRHS1''' = [https://music-ir.org/mirex/results/2009/chord/task1/RRHS1.task1.overlap.results.csv Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh]<br /> |
| − | '''RSHS2''' = [https://music-ir.org/mirex/2009 | + | '''RSHS2''' = [https://music-ir.org/mirex/results/2009/chord/task1/RRHS2.task1.overlap.results.csv Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh]<br /> |
| − | '''RSHS3''' = [https://music-ir.org/mirex/2009 | + | '''RSHS3''' = [https://music-ir.org/mirex/results/2009/chord/task1/RRHS3.task1.overlap.results.csv Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh]<br /> |
==== Task2 ==== | ==== Task2 ==== | ||
| − | '''RUSUSL''' = [https://music-ir.org/mirex/2009 | + | '''RUSUSL''' = [https://music-ir.org/mirex/results/2009/chord/task2/RUSUSL.task2.overlap.results.csv J.T.Reed,Yushi Ueda,S.Siniscalchi,Yuki Uchiyama,Shigeki Sagayama,C.H.Lee]<br /> |
| − | '''WEJ1''' = [https://music-ir.org/mirex/2009 | + | '''WEJ1''' = [https://music-ir.org/mirex/results/2009/chord/task2/WEJ1.task2.overlap.results.csv Adrian Weller, Daniel Ellis, Tony Jebara]<br /> |
| − | '''WEJ2''' = [https://music-ir.org/mirex/2009 | + | '''WEJ2''' = [https://music-ir.org/mirex/results/2009/chord/task2/WEJ2.task2.overlap.results.csv Adrian Weller, Daniel Ellis, Tony Jebara]<br /> |
| − | '''WEJ3''' = [https://music-ir.org/mirex/2009 | + | '''WEJ3''' = [https://music-ir.org/mirex/results/2009/chord/task2/WEJ3.task2.overlap.results.csv Adrian Weller, Daniel Ellis, Tony Jebara]<br /> |
| − | '''WEJ4''' = [https://music-ir.org/mirex/2009 | + | '''WEJ4''' = [https://music-ir.org/mirex/results/2009/chord/task2/WEJ4.task2.overlap.results.csv Adrian Weller, Daniel Ellis, Tony Jebara]<br /> |
== Evaluations by Christopher Harte's Metrics == | == Evaluations by Christopher Harte's Metrics == | ||
TBA | TBA | ||
Latest revision as of 21:44, 13 May 2010
Contents
Introduction
These are the results for the 2009 running of the Audio Chord Detection task set. For background information about this task set please refer to the 2009:Audio Chord Detection page.
Task Descriptions
Beatles data donated by Christopher Harte and extra dataset of around 37 songs donated by Matthias Mauch was used. 7 songs were excluded from the list where the average score for these songs across participants were less than 0.25
Task 1 (Pretrained Systems) Go to Task 1 Results:
Task 2 (Train-Test Systems) Go to Task 2 Results:
System trained on ~2/3 of the dataset and tested on ~1/3. Folds were divided such that 2/3 of each album was represented in the train list and 1/3 in the test list.
Overlap score was calculated as the ratio between the overlap of the ground truth and detected chords and ground truth duration. Also a secondary overlap score was calculated by ignoring the major-minor variations of the detected chord (e.g., C major == C minor, etc.). The weighted average is computed across the songs by weighting each score by its the duration.
General Legend
Team ID
CH = Christopher Harte
DE = Dan Ellis
KO1 = Maksim Khadkevich, Maurizio Omologo
KO2 = Maksim Khadkevich, Maurizio Omologo
MD = Matthias Mauch, Katy Noland, Simon Dixon
OGF1 = Laurent Oudre, Yves Grenier,Cedric Fevotte (majmin)
OGF2 = Laurent Oudre, Yves Grenier,Cedric Fevotte (majmin7)
PP = Helene Papadopoulos, Geoffroy Peeters
PVM1 = Johan Pauwels, Matthias Varewyck,Jean-Pierre Martens
PVM2 = Johan Pauwels, Matthias Varewyck,Jean-Pierre Martens
RRHS1 = Thomas Rocher, Matthias Robine, Pierre Hanna,Robert Strandh
RSHS2 = Thomas Rocher, Matthias Robine, Pierre Hanna,Robert Strandh
RSHS3 = Thomas Rocher, Matthias Robine, Pierre Hanna,Robert Strandh
RUSUSL = J.T.Reed,Yushi Ueda,S.Siniscalchi,Yuki Uchiyama,Shigeki Sagayama,C.H.Lee
WEJ1 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ2 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ3 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ4 = Adrian Weller, Daniel Ellis, Tony Jebara
Overall Summary Results
Task 1 Results
Task 1 Overall Results
| CH | DE | KO1 | KO2 | MD | OGF1 | OGF2 | PP | PVM1 | PVM2 | RRHS1 | RRHS2 | RRHS3 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Wt. Ave. Overlap Score | 0.576 | 0.697 | 0.697 | 0.708 | 0.712 | 0.706 | 0.711 | 0.673 | 0.682 | 0.654 | 0.629 | 0.545 | 0.538 |
| Wt. Ave. Overlap Score (Merged maj/min) | 0.625 | 0.731 | 0.734 | 0.741 | 0.748 | 0.770 | 0.777 | 0.715 | 0.710 | 0.698 | 0.688 | 0.600 | 0.688 |
Task 1 Run Times
TBA file /nema-raid/www/mirex/results/2009/chord/task1/task1_runtimes.csv not found
Task 1 Results Per Artist/Album
2009:Audio Chord Detection Results Per Album#Task 1
Task 1 Friedman's Test for Significant Differences
The Tukey-Kramer HSD multi-comparison data below was generate using the following MATLAB instruction.
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);
| TeamID | TeamID | Lowerbound | Mean | Upperbound | Significance |
|---|---|---|---|---|---|
| OGF2 | MD | -1.1304 | 0.1286 | 1.3876 | FALSE |
| OGF2 | OGF1 | -1.0638 | 0.1952 | 1.4543 | FALSE |
| OGF2 | KO2 | -0.7900 | 0.4690 | 1.7281 | FALSE |
| OGF2 | DE | -0.2638 | 0.9952 | 2.2543 | FALSE |
| OGF2 | KO1 | -0.0185 | 1.2405 | 2.4995 | FALSE |
| OGF2 | PP | 0.3743 | 1.6333 | 2.8924 | TRUE |
| OGF2 | PVM1 | 0.1981 | 1.4571 | 2.7162 | TRUE |
| OGF2 | PVM2 | 1.4029 | 2.6619 | 3.9209 | TRUE |
| OGF2 | RRHS1 | 2.6981 | 3.9571 | 5.2162 | TRUE |
| OGF2 | CH | 4.5767 | 5.8357 | 7.0947 | TRUE |
| OGF2 | RRHS2 | 5.8696 | 7.1286 | 8.3876 | TRUE |
| OGF2 | RRHS3 | 5.9719 | 7.2310 | 8.4900 | TRUE |
| MD | OGF1 | -1.1924 | 0.0667 | 1.3257 | FALSE |
| MD | KO2 | -0.9185 | 0.3405 | 1.5995 | FALSE |
| MD | DE | -0.3924 | 0.8667 | 2.1257 | FALSE |
| MD | KO1 | -0.1471 | 1.1119 | 2.3709 | FALSE |
| MD | PP | 0.2457 | 1.5048 | 2.7638 | TRUE |
| MD | PVM1 | 0.0696 | 1.3286 | 2.5876 | TRUE |
| MD | PVM2 | 1.2743 | 2.5333 | 3.7924 | TRUE |
| MD | RRHS1 | 2.5696 | 3.8286 | 5.0876 | TRUE |
| MD | CH | 4.4481 | 5.7071 | 6.9662 | TRUE |
| MD | RRHS2 | 5.7410 | 7.0000 | 8.2590 | TRUE |
| MD | RRHS3 | 5.8434 | 7.1024 | 8.3614 | TRUE |
| OGF1 | KO2 | -0.9852 | 0.2738 | 1.5328 | FALSE |
| OGF1 | DE | -0.4590 | 0.8000 | 2.0590 | FALSE |
| OGF1 | KO1 | -0.2138 | 1.0452 | 2.3043 | FALSE |
| OGF1 | PP | 0.1791 | 1.4381 | 2.6971 | TRUE |
| OGF1 | PVM1 | 0.0029 | 1.2619 | 2.5209 | TRUE |
| OGF1 | PVM2 | 1.2076 | 2.4667 | 3.7257 | TRUE |
| OGF1 | RRHS1 | 2.5029 | 3.7619 | 5.0209 | TRUE |
| OGF1 | CH | 4.3815 | 5.6405 | 6.8995 | TRUE |
| OGF1 | RRHS2 | 5.6743 | 6.9333 | 8.1924 | TRUE |
| OGF1 | RRHS3 | 5.7767 | 7.0357 | 8.2947 | TRUE |
| KO2 | DE | -0.7328 | 0.5262 | 1.7852 | FALSE |
| KO2 | KO1 | -0.4876 | 0.7714 | 2.0304 | FALSE |
| KO2 | PP | -0.0947 | 1.1643 | 2.4233 | FALSE |
| KO2 | PVM1 | -0.2709 | 0.9881 | 2.2471 | FALSE |
| KO2 | PVM2 | 0.9338 | 2.1929 | 3.4519 | TRUE |
| KO2 | RRHS1 | 2.2291 | 3.4881 | 4.7471 | TRUE |
| KO2 | CH | 4.1076 | 5.3667 | 6.6257 | TRUE |
| KO2 | RRHS2 | 5.4005 | 6.6595 | 7.9185 | TRUE |
| KO2 | RRHS3 | 5.5029 | 6.7619 | 8.0209 | TRUE |
| DE | KO1 | -1.0138 | 0.2452 | 1.5043 | FALSE |
| DE | PP | -0.6209 | 0.6381 | 1.8971 | FALSE |
| DE | PVM1 | -0.7971 | 0.4619 | 1.7209 | FALSE |
| DE | PVM2 | 0.4076 | 1.6667 | 2.9257 | TRUE |
| DE | RRHS1 | 1.7029 | 2.9619 | 4.2209 | TRUE |
| DE | CH | 3.5815 | 4.8405 | 6.0995 | TRUE |
| DE | RRHS2 | 4.8743 | 6.1333 | 7.3924 | TRUE |
| DE | RRHS3 | 4.9767 | 6.2357 | 7.4947 | TRUE |
| KO1 | PP | -0.8662 | 0.3929 | 1.6519 | FALSE |
| KO1 | PVM1 | -1.0424 | 0.2167 | 1.4757 | FALSE |
| KO1 | PVM2 | 0.1624 | 1.4214 | 2.6804 | TRUE |
| KO1 | RRHS1 | 1.4576 | 2.7167 | 3.9757 | TRUE |
| KO1 | CH | 3.3362 | 4.5952 | 5.8543 | TRUE |
| KO1 | RRHS2 | 4.6291 | 5.8881 | 7.1471 | TRUE |
| KO1 | RRHS3 | 4.7315 | 5.9905 | 7.2495 | TRUE |
| PP | PVM1 | -1.4352 | -0.1762 | 1.0828 | FALSE |
| PP | PVM2 | -0.2304 | 1.0286 | 2.2876 | FALSE |
| PP | RRHS1 | 1.0648 | 2.3238 | 3.5828 | TRUE |
| PP | CH | 2.9434 | 4.2024 | 5.4614 | TRUE |
| PP | RRHS2 | 4.2362 | 5.4952 | 6.7543 | TRUE |
| PP | RRHS3 | 4.3386 | 5.5976 | 6.8566 | TRUE |
| PVM1 | PVM2 | -0.0543 | 1.2048 | 2.4638 | FALSE |
| PVM1 | RRHS1 | 1.2410 | 2.5000 | 3.7590 | TRUE |
| PVM1 | CH | 3.1196 | 4.3786 | 5.6376 | TRUE |
| PVM1 | RRHS2 | 4.4124 | 5.6714 | 6.9304 | TRUE |
| PVM1 | RRHS3 | 4.5148 | 5.7738 | 7.0328 | TRUE |
| PVM2 | RRHS1 | 0.0362 | 1.2952 | 2.5543 | TRUE |
| PVM2 | CH | 1.9148 | 3.1738 | 4.4328 | TRUE |
| PVM2 | RRHS2 | 3.2076 | 4.4667 | 5.7257 | TRUE |
| PVM2 | RRHS3 | 3.3100 | 4.5690 | 5.8281 | TRUE |
| RRHS1 | CH | 0.6196 | 1.8786 | 3.1376 | TRUE |
| RRHS1 | RRHS2 | 1.9124 | 3.1714 | 4.4304 | TRUE |
| RRHS1 | RRHS3 | 2.0148 | 3.2738 | 4.5328 | TRUE |
| CH | RRHS2 | 0.0338 | 1.2929 | 2.5519 | TRUE |
| CH | RRHS3 | 0.1362 | 1.3952 | 2.6543 | TRUE |
| RRHS2 | RRHS3 | -1.1566 | 0.1024 | 1.3614 | FALSE |
Task 2 Results
Task 2 Overall Results
| RUSUSL | WEJ1 | WEJ2 | WEJ3 | WEJ4 | |
|---|---|---|---|---|---|
| Wt. Ave. Overlap Score | 0.701 | 0.704 | 0.723 | 0.723 | 0.742 |
| Wt. Ave. Overlap Score (Merged maj/min) | 0.760 | 0.743 | 0.762 | 0.760 | 0.777 |
Task 2 Run Times
TBA file /nema-raid/www/mirex/results/2009/chord/task2/task2_runtimes.csv not found
Task 2 Results Per Artist/Album
2009:Audio Chord Detection Results Per Album#Task 2
Task 2 Friedman's Test for Significant Differences
The Tukey-Kramer HSD multi-comparison data below was generate using the following MATLAB instruction.
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer', 'estimate', 'friedman', 'alpha', 0.05);
| TeamID | TeamID | Lowerbound | Mean | Upperbound | Significance |
|---|---|---|---|---|---|
| WEJ4 | WEJ3 | 0.2936 | 0.7143 | 1.1349 | TRUE |
| WEJ4 | WEJ2 | 0.2341 | 0.6548 | 1.0754 | TRUE |
| WEJ4 | WEJ1 | 1.0055 | 1.4262 | 1.8468 | TRUE |
| WEJ4 | RUSUSL | 0.7960 | 1.2167 | 1.6373 | TRUE |
| WEJ3 | WEJ2 | -0.4802 | -0.0595 | 0.3611 | FALSE |
| WEJ3 | WEJ1 | 0.2913 | 0.7119 | 1.1326 | TRUE |
| WEJ3 | RUSUSL | 0.0817 | 0.5024 | 0.9230 | TRUE |
| WEJ2 | WEJ1 | 0.3508 | 0.7714 | 1.1921 | TRUE |
| WEJ2 | RUSUSL | 0.1413 | 0.5619 | 0.9826 | TRUE |
| WEJ1 | RUSUSL | -0.6302 | -0.2095 | 0.2111 | FALSE |
Individual Results
Task1
CH = Christopher Harte
DE = Dan Ellis
KO1 = Maksim Khadkevich, Maurizio Omologo
KO2 = Maksim Khadkevich, Maurizio Omologo
MD = Matthias Mauch, Simon Dixon
OGF1 = Laurent Oudre,Yves Grenier,Cedric Fevotte
OGF2 = Laurent Oudre,Yves Grenier,Cedric Fevotte
PP = Helene Papadopoulos, Geoffroy Peeters
PVM1 = Johan Pauwels,Matthias Varewyck,Jean-Pierre Martens
PVM2 = Johan Pauwels,Matthias Varewyck,Jean-Pierre Martens
RRHS1 = Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh
RSHS2 = Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh
RSHS3 = Thomas Rocher, Matthias Robine,Pierre Hanna,Robert Strandh
Task2
RUSUSL = J.T.Reed,Yushi Ueda,S.Siniscalchi,Yuki Uchiyama,Shigeki Sagayama,C.H.Lee
WEJ1 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ2 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ3 = Adrian Weller, Daniel Ellis, Tony Jebara
WEJ4 = Adrian Weller, Daniel Ellis, Tony Jebara
Evaluations by Christopher Harte's Metrics
TBA
