Difference between revisions of "2009:Audio Melody Extraction Results"
IMIRSELBot (talk | contribs) m (Robot: Automated text replacement (-<csv([^>]*)> +<csv\1>2009/)) |
(→Team ID) |
||
(2 intermediate revisions by one other user not shown) | |||
Line 7: | Line 7: | ||
==== Team ID ==== | ==== Team ID ==== | ||
− | '''CL1''' = [https://music-ir.org/mirex/2009 | + | '''CL1''' = [https://www.music-ir.org/mirex/abstracts/2009/CLmelody.pdf Chuan Cao, Ming Li]<br /> |
− | '''CL2''' = [https://music-ir.org/mirex/2009 | + | '''CL2''' = [https://www.music-ir.org/mirex/abstracts/2009/CLmelody.pdf Chuan Cao, Ming Li]<br /> |
− | '''DR1''' = [https://music-ir.org/mirex/2009 | + | '''DR1''' = [https://www.music-ir.org/mirex/abstracts/2009/DR.pdf Jean-Louis Durrieu, Gaël Richard, Bertrand David (GSMM)]<br /> |
− | '''DR2''' = [https://music-ir.org/mirex/2009 | + | '''DR2''' = [https://www.music-ir.org/mirex/abstracts/2009/DR.pdf Jean-Louis Durrieu, Gaël Richard, Bertrand David (SIMM)]<br /> |
− | '''HJC1''' = [https://music-ir.org/mirex/2009 | + | '''HJC1''' = [https://www.music-ir.org/mirex/abstracts/2009/HJC.pdf Chao-Ling Hsu, Jyh-Shing Roger Jang, Liang-Yu Chen (DP)]<br /> |
− | '''HJC2''' = [https://music-ir.org/mirex/2009 | + | '''HJC2''' = [https://www.music-ir.org/mirex/abstracts/2009/HJC.pdf Chao-Ling Hsu, Jyh-Shing Roger Jang, Liang-Yu Chen (HMM)]<br /> |
− | '''JJY''' = [https://music-ir.org/mirex/2009 | + | '''JJY''' = [https://www.music-ir.org/mirex/abstracts/2009/JJY.pdf Sihyun Joo, Seokhwan Jo, Chang D. Yoo]<br /> |
− | '''KD''' = [https://music-ir.org/mirex/2009 | + | '''KD''' = [https://www.music-ir.org/mirex/abstracts/2009/KD.pdf Karin Dressler]<br /> |
'''MW''' = [http://morwen.dk/university/master-thesis/Melody-Extraction-MIREX2009_Morten-Wendelboe.pdf Morten Wendelboe]<br /> | '''MW''' = [http://morwen.dk/university/master-thesis/Melody-Extraction-MIREX2009_Morten-Wendelboe.pdf Morten Wendelboe]<br /> | ||
− | '''PC''' = [https://music-ir.org/mirex/2009 | + | '''PC''' = [https://www.music-ir.org/mirex/abstracts/2009/PC.txt Pablo Cancela]<br /> |
− | '''RR''' = [https://music-ir.org/mirex/2009 | + | '''RR''' = [https://www.music-ir.org/mirex/abstracts/2009/RR.pdf Vishweshwara Rao,Preeti Rao]<br /> |
− | '''TOOS''' = [https://music-ir.org/mirex/2009 | + | '''TOOS''' = [https://www.music-ir.org/mirex/abstracts/2009/TOOS.pdf Hideyuki Tachibana, Takuma Ono, Nobutaka Ono, Shigeki Sagayama]<br /> |
====Table Headings==== | ====Table Headings==== |
Latest revision as of 16:27, 23 July 2010
Contents
- 1 Introduction
- 2 Overall Summary Results
- 2.1 MIREX 2009 Audio Melody Extraction Overall Summary results - Unweighted Avg. of all Datasets
- 2.2 MIREX 2009 Audio Melody Extraction Overall Summary results - Weighted (by Number of Files) Avg. of all Datasets
- 2.3 MIREX 2009 Audio Melody Extraction Summary results - MIREX 2009 Dataset - -5dB mix
- 2.4 MIREX 2009 Audio Melody Extraction Summary results - MIREX 2009 Dataset - 0dB mix
- 2.5 MIREX 2009 Audio Melody Extraction Summary results - MIREX 2009 Dataset - +5dB mix
- 2.6 MIREX 2009 Audio Melody Extraction Summary results - MIREX 2008 Dataset - All
- 2.7 MIREX 2009 Audio Melody Extraction Summary results - MIREX 2005 Dataset - vocal
- 2.8 MIREX 2009 Audio Melody Extraction Summary results - MIREX 2005 Dataset - nonvocal
- 2.9 MIREX 2009 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All
- 2.10 MIREX 2009 Audio Melody Extraction Summary results - ADC 2004 Dataset - vocal
- 2.11 MIREX 2009 Audio Melody Extraction Summary results - ADC 2004 Dataset - nonvocal
- 2.12 MIREX 2009 Audio Melody Extraction Summary results - ADC 2004 Dataset - All
- 2.13 MIREX 2009 Audio Melody Extraction Runtime Data
Introduction
These are the results for the 2009 running of the Audio Melody Extraction task set. For background information about this task set please refer to the 2009:Audio Melody Extraction page. There are multiple datasets for this task. ADC04 consists of 20 pieces with vocal and nonvocal melodies. MIREX05 consists of 25 pieces with vocal and nonvocal melodies. MIREX08 consists of 8 pieces all with vocal melodies. MIREX09 consists of 374 pieces all with vocal melodies. MIREX09 is mixed with three different Melodic Voice - to accompaniment ratios. +5dB, 0dB, and -5 dB RMS.
General Legend
Team ID
CL1 = Chuan Cao, Ming Li
CL2 = Chuan Cao, Ming Li
DR1 = Jean-Louis Durrieu, Gaël Richard, Bertrand David (GSMM)
DR2 = Jean-Louis Durrieu, Gaël Richard, Bertrand David (SIMM)
HJC1 = Chao-Ling Hsu, Jyh-Shing Roger Jang, Liang-Yu Chen (DP)
HJC2 = Chao-Ling Hsu, Jyh-Shing Roger Jang, Liang-Yu Chen (HMM)
JJY = Sihyun Joo, Seokhwan Jo, Chang D. Yoo
KD = Karin Dressler
MW = Morten Wendelboe
PC = Pablo Cancela
RR = Vishweshwara Rao,Preeti Rao
TOOS = Hideyuki Tachibana, Takuma Ono, Nobutaka Ono, Shigeki Sagayama
Table Headings
Vx Recall = Voicing Detection
Vx False Alm = Voicing False Alarm
Raw pitch = Raw Pitch Accuracy
Raw Chroma = Raw Chroma Accuracy
Overall Acc = Overall Acuuracy
Overall Summary Results
MIREX 2009 Audio Melody Extraction Overall Summary results - Unweighted Avg. of all Datasets
Participant | Vx Recall(%) | Vx False Alm(%) | Raw Pitch(%) | Raw Chroma(%) | Overall Acc(%) |
---|---|---|---|---|---|
cl1 | 93.0122 | 80.7121 | 63.4537 | 66.2928 | 52.192 |
cl2 | 80.2955 | 57.4239 | 63.4537 | 66.2928 | 55.1912 |
dr1 | 92.4039 | 51.7437 | 74.4454 | 76.8213 | 66.8572 |
dr2 | 87.687 | 41.2182 | 72.0925 | 75.7223 | 66.172 |
hjc1 | 43.6213 | 9.7101 | 66.1219 | 72.581 | 50.4858 |
hjc2 | 43.6213 | 9.7101 | 51.1347 | 67.1245 | 49.0116 |
jjy | 61.023 | 29.3892 | 73.3253 | 79.6779 | 56.6353 |
kd | 90.9304 | 40.9859 | 80.5833 | 82.5201 | 73.3523 |
mw | 99.9724 | 98.6594 | 73.4386 | 77.4986 | 55.0722 |
pc | 79.3165 | 40.292 | 64.1009 | 65.8397 | 62.8812 |
rr | 91.28 | 51.1058 | 72.2126 | 76.3331 | 65.2212 |
toos | 99.8687 | 98.3127 | 75.0514 | 80.3403 | 55.0774 |
MIREX 2009 Audio Melody Extraction Overall Summary results - Weighted (by Number of Files) Avg. of all Datasets
Note, because the MIREX09 dataset is so large in comparison, the weighted average is heavily weighted to performance on this dataset.
Participant | Vx Recall(%) | Vx False Alm(%) | Raw Pitch(%) | Raw Chroma(%) | Overall Acc(%) |
---|---|---|---|---|---|
cl1 | 92.3264 | 83.1113 | 58.9205 | 62.7661 | 44.3575 |
cl2 | 77.2452 | 59.2516 | 58.9205 | 62.7661 | 49.5138 |
dr1 | 91.7457 | 53.7476 | 68.6411 | 71.3884 | 60.0492 |
dr2 | 87.2511 | 45.2327 | 65.3401 | 69.8261 | 59.5668 |
hjc1 | 37.8763 | 2.8033 | 68.4293 | 72.0766 | 54.9707 |
hjc2 | 37.8763 | 2.8033 | 50.4366 | 66.9551 | 54.1855 |
jjy | 38.0818 | 17.8244 | 73.0217 | 77.7374 | 48.6962 |
kd | 90.5445 | 47.2188 | 77.5947 | 79.5145 | 66.7341 |
mw | 99.9896 | 99.3837 | 66.4069 | 70.34 | 43.6651 |
pc | 73.4387 | 42.4767 | 51.778 | 54.1777 | 52.5412 |
rr | 89.2142 | 49.0418 | 67.4012 | 70.4479 | 60.618 |
toos | 99.8103 | 99.1475 | 80.0558 | 83.7591 | 52.711 |
MIREX 2009 Audio Melody Extraction Summary results - MIREX 2009 Dataset - -5dB mix
Participant | Vx Recall(%) | Vx False Alm(%) | Raw Pitch(%) | Raw Chroma(%) | Overall Acc(%) |
---|---|---|---|---|---|
cl1 | 90.6211 | 84.8568 | 45.3909 | 50.9368 | 34.504 |
cl2 | 72.8248 | 61.2662 | 45.3909 | 50.9368 | 39.9455 |
dr1 | 88.5333 | 67.0915 | 53.7796 | 58.0902 | 45.5482 |
dr2 | 85.3196 | 60.2824 | 50.5318 | 57.5288 | 44.7759 |
hjc1 | 8.5966 | 1.1051 | 48.658 | 54.5224 | 38.4559 |
hjc2 | 8.5966 | 1.1051 | 21.4502 | 44.5494 | 37.5497 |
jjy | 39.0921 | 26.4556 | 58.5304 | 64.7866 | 42.2335 |
kd | 86.3897 | 60.1029 | 62.4877 | 66.2816 | 51.6864 |
mw | 99.992 | 99.4688 | 53.0621 | 58.7232 | 34.2691 |
pc | 67.6162 | 48.1973 | 37.3777 | 40.5081 | 41.6186 |
rr | 92.7872 | 75.7569 | 54.6785 | 58.7592 | 43.3962 |
toos | 99.9829 | 99.4185 | 74.8896 | 78.5338 | 48.6449 |
MIREX 2009 Audio Melody Extraction Summary results - MIREX 2009 Dataset - 0dB mix
Participant | Vx Recall(%) | Vx False Alm(%) | Raw Pitch(%) | Raw Chroma(%) | Overall Acc(%) |
---|---|---|---|---|---|
cl1 | 92.4858 | 83.5749 | 59.138 | 62.9508 | 43.9659 |
cl2 | 77.2085 | 59.7352 | 59.138 | 62.9508 | 49.2294 |
dr1 | 91.8716 | 55.3555 | 69.8804 | 72.5138 | 60.1294 |
dr2 | 87.3985 | 47.3422 | 66.549 | 70.7923 | 59.5076 |
hjc1 | 34.1722 | 1.7909 | 72.6577 | 75.2906 | 53.1752 |
hjc2 | 34.1722 | 1.7909 | 51.6871 | 70.002 | 51.7469 |
jjy | 38.906 | 19.4063 | 75.9354 | 80.2461 | 49.686 |
kd | 91.1846 | 47.7842 | 80.4565 | 81.8811 | 68.2237 |
mw | 99.992 | 99.4688 | 67.2905 | 71.0018 | 43.6365 |
pc | 73.1175 | 43.4773 | 50.8895 | 53.3672 | 51.5001 |
rr | 88.8091 | 50.7595 | 68.6242 | 71.3714 | 60.7733 |
toos | 99.9829 | 99.4185 | 82.2943 | 85.7474 | 53.5623 |
MIREX 2009 Audio Melody Extraction Summary results - MIREX 2009 Dataset - +5dB mix
Participant | Vx Recall(%) | Vx False Alm(%) | Raw Pitch(%) | Raw Chroma(%) | Overall Acc(%) |
---|---|---|---|---|---|
cl1 | 93.6769 | 82.0974 | 70.2637 | 72.6758 | 51.7453 |
cl2 | 80.8723 | 57.6872 | 70.2637 | 72.6758 | 57.1658 |
dr1 | 94.785 | 39.3475 | 80.8947 | 82.2161 | 72.7971 |
dr2 | 89.1418 | 29.2436 | 77.2989 | 79.6474 | 72.7917 |
hjc1 | 69.3829 | 2.9927 | 84.8561 | 86.3581 | 74.7639 |
hjc2 | 69.3829 | 2.9927 | 78.3801 | 86.5939 | 74.9723 |
jjy | 29.378 | 3.9263 | 84.3853 | 87.6795 | 51.7425 |
kd | 94.1297 | 36.2944 | 89.1898 | 89.6585 | 78.4061 |
mw | 99.992 | 99.4688 | 77.0268 | 79.3507 | 50.029 |
pc | 78.2207 | 37.4586 | 63.6794 | 65.4185 | 61.5177 |
rr | 85.4713 | 19.6651 | 77.8827 | 79.6838 | 76.7448 |
toos | 99.4507 | 98.8844 | 84.8473 | 88.289 | 55.6746 |
MIREX 2009 Audio Melody Extraction Summary results - MIREX 2008 Dataset - All
Participant | Vx Recall(%) | Vx False Alm(%) | Raw Pitch(%) | Raw Chroma(%) | Overall Acc(%) |
---|---|---|---|---|---|
cl1 | 93.3948 | 90.5445 | 50.8092 | 51.3331 | 45.3441 |
cl2 | 84.4221 | 66.1472 | 50.8092 | 51.3331 | 46.8145 |
dr1 | 96.9986 | 51.0862 | 88.0057 | 88.1693 | 81.1762 |
dr2 | 94.1888 | 39.2986 | 86.5807 | 86.808 | 80.0331 |
hjc1 | 55.9319 | 7.7332 | 67.5624 | 74.8665 | 48.0678 |
hjc2 | 55.9319 | 7.7332 | 60.8391 | 74.8106 | 46.5056 |
jjy | 85.1639 | 38.9125 | 68.2963 | 81.8753 | 61.1604 |
kd | 95.0744 | 53.3054 | 87.82 | 88.8227 | 80.6539 |
mw | 100 | 99.8667 | 85.9869 | 88.8665 | 73.4976 |
pc | 94.0546 | 66.0993 | 81.8281 | 81.9825 | 73.6415 |
rr | 94.8175 | 48.1976 | 86.161 | 86.6712 | 78.9667 |
toos | 100 | 99.2636 | 79.7606 | 83.6605 | 68.5002 |
MIREX 2009 Audio Melody Extraction Summary results - MIREX 2005 Dataset - vocal
Participant | Vx Recall(%) | Vx False Alm(%) | Raw Pitch(%) | Raw Chroma(%) | Overall Acc(%) |
---|---|---|---|---|---|
cl1 | 91.2267 | 67.7071 | 70.807 | 73.924 | 59.7095 |
cl2 | 80.9512 | 44.8062 | 70.807 | 73.924 | 64.4609 |
dr1 | 93.7543 | 53.5255 | 76.1145 | 77.7138 | 66.9613 |
dr2 | 88.0635 | 38.0207 | 70.9258 | 75.9161 | 66.5175 |
hjc1 | 65.8441 | 19.8206 | 62.6594 | 73.4868 | 54.8538 |
hjc2 | 65.8441 | 19.8206 | 54.1294 | 69.4221 | 52.2905 |
jjy | 88.8546 | 41.9813 | 76.2696 | 79.3236 | 66.3123 |
kd | 82.6017 | 15.2554 | 77.4622 | 80.8218 | 76.962 |
mw | 99.9345 | 99.7947 | 75.7398 | 80.3791 | 53.7323 |
pc | 75.6436 | 21.0879 | 71.7068 | 72.5089 | 70.465 |
rr | 92.908 | 56.3639 | 75.9506 | 79.1084 | 65.7745 |
toos | 99.883 | 99.5427 | 73.4258 | 77.8075 | 52.1017 |
MIREX 2009 Audio Melody Extraction Summary results - MIREX 2005 Dataset - nonvocal
Participant | Vx Recall(%) | Vx False Alm(%) | Raw Pitch(%) | Raw Chroma(%) | Overall Acc(%) |
---|---|---|---|---|---|
cl1 | 94.1197 | 84.9564 | 68.7094 | 72.7874 | 63.2993 |
cl2 | 78.3576 | 69.1015 | 68.7094 | 72.7874 | 59.2786 |
dr1 | 88.5869 | 61.4615 | 66.6591 | 74.3354 | 63.7449 |
dr2 | 82.038 | 56.025 | 69.5099 | 76.0359 | 62.9567 |
hjc1 | 16.0485 | 40.823 | 52.8876 | 66.1191 | 16.5249 |
hjc2 | 16.0485 | 40.823 | 25.8824 | 48.5695 | 14.5299 |
jjy | 83.2623 | 69.2165 | 57.5975 | 71.4552 | 48.4402 |
kd | 95.2984 | 63.5702 | 74.6221 | 80.9244 | 71.0659 |
mw | 99.8919 | 99.9457 | 73.7136 | 81.1205 | 66.1035 |
pc | 82.8826 | 48.6806 | 61.3362 | 66.5582 | 59.4657 |
rr | 95.3665 | 78.4275 | 56.781 | 71.6471 | 51.7068 |
toos | 99.8969 | 100 | 56.9256 | 67.3221 | 50.6871 |
MIREX 2009 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All
Participant | Vx Recall(%) | Vx False Alm(%) | Raw Pitch(%) | Raw Chroma(%) | Overall Acc(%) |
---|---|---|---|---|---|
cl1 | 92.2682 | 73.9168 | 70.0518 | 73.5148 | 61.0018 |
cl2 | 80.0175 | 53.5525 | 70.0518 | 73.5148 | 62.5953 |
dr1 | 91.894 | 56.3825 | 72.7105 | 76.4976 | 65.8034 |
dr2 | 85.8943 | 44.5023 | 70.4161 | 75.9592 | 65.2356 |
hjc1 | 47.9177 | 27.3815 | 59.1416 | 70.8344 | 41.0554 |
hjc2 | 47.9177 | 27.3815 | 43.9605 | 61.9152 | 38.6967 |
jjy | 86.8414 | 51.786 | 69.5477 | 76.491 | 59.8783 |
kd | 87.1725 | 32.6487 | 76.4398 | 80.8588 | 74.8394 |
mw | 99.9191 | 99.8491 | 75.0104 | 80.646 | 58.1859 |
pc | 78.2496 | 31.0213 | 67.9734 | 70.3666 | 66.5052 |
rr | 93.7931 | 64.3068 | 69.0496 | 76.4223 | 60.7101 |
toos | 99.888 | 99.7073 | 67.4857 | 74.0327 | 51.5925 |
MIREX 2009 Audio Melody Extraction Summary results - ADC 2004 Dataset - vocal
Participant | Vx Recall(%) | Vx False Alm(%) | Raw Pitch(%) | Raw Chroma(%) | Overall Acc(%) |
---|---|---|---|---|---|
cl1 | 95.8377 | 79.3617 | 85.6252 | 86.2052 | 75.1569 |
cl2 | 88.0757 | 51.9066 | 85.6252 | 86.2052 | 75.3229 |
dr1 | 92.3581 | 49.2541 | 86.9605 | 87.398 | 79.9939 |
dr2 | 86.6218 | 28.4689 | 83.2614 | 85.1916 | 78.0742 |
hjc1 | 50.0165 | 25.3791 | 63.1101 | 74.101 | 48.765 |
hjc2 | 50.0165 | 25.3791 | 46.5192 | 64.4711 | 44.9189 |
jjy | 85.8299 | 39.0501 | 81.9596 | 85.7982 | 74.6814 |
kd | 91.6258 | 14.9754 | 85.9698 | 86.4237 | 85.8694 |
mw | 99.9368 | 98.4374 | 83.1351 | 86.593 | 70.9614 |
pc | 88.781 | 19.3581 | 86.9624 | 87.5452 | 85.9451 |
rr | 92.8267 | 59.1767 | 81.446 | 88.0381 | 73.7674 |
toos | 99.9052 | 97.7855 | 59.7683 | 72.1289 | 50.745 |
MIREX 2009 Audio Melody Extraction Summary results - ADC 2004 Dataset - nonvocal
Participant | Vx Recall(%) | Vx False Alm(%) | Raw Pitch(%) | Raw Chroma(%) | Overall Acc(%) |
---|---|---|---|---|---|
cl1 | 95.3094 | 54.1628 | 84.2344 | 86.5563 | 78.7418 |
cl2 | 83.9553 | 37.5275 | 84.2344 | 86.5563 | 75.5083 |
dr1 | 87.315 | 29.117 | 73.0634 | 77.5043 | 69.2323 |
dr2 | 80.5139 | 23.8977 | 78.0547 | 81.2086 | 69.6093 |
hjc1 | 39.2916 | 5.0734 | 64.9745 | 72.8834 | 45.3448 |
hjc2 | 39.2916 | 5.0734 | 56.4498 | 65.483 | 44.1171 |
jjy | 88.1466 | 31.0457 | 85.2026 | 88.7742 | 75.7553 |
kd | 91.6407 | 16.9873 | 88.8104 | 89.4088 | 86.9568 |
mw | 99.9436 | 86.9298 | 80.9344 | 86.1186 | 70.596 |
pc | 78.4299 | 9.7087 | 76.6996 | 77.1712 | 77.3419 |
rr | 90.7642 | 31.1078 | 70.0296 | 80.6697 | 66.189 |
toos | 99.9114 | 86.2809 | 62.9242 | 71.2532 | 55.1078 |
MIREX 2009 Audio Melody Extraction Summary results - ADC 2004 Dataset - All
Participant | Vx Recall(%) | Vx False Alm(%) | Raw Pitch(%) | Raw Chroma(%) | Overall Acc(%) |
---|---|---|---|---|---|
cl1 | 95.6264 | 69.2822 | 85.0689 | 86.3457 | 76.5909 |
cl2 | 86.4276 | 46.155 | 85.0689 | 86.3457 | 75.397 |
dr1 | 90.3409 | 41.1992 | 81.4017 | 83.4406 | 75.6892 |
dr2 | 84.1787 | 26.6404 | 81.1787 | 83.5984 | 74.6882 |
hjc1 | 45.7265 | 17.2569 | 63.8558 | 73.614 | 47.3969 |
hjc2 | 45.7265 | 17.2569 | 50.4915 | 64.8759 | 44.5982 |
jjy | 86.7566 | 35.8484 | 83.2568 | 86.9886 | 75.111 |
kd | 91.6318 | 15.7801 | 87.106 | 87.6177 | 86.3043 |
mw | 99.9395 | 93.8344 | 82.2548 | 86.4033 | 70.8152 |
pc | 84.6406 | 15.4983 | 82.8573 | 83.3956 | 82.5038 |
rr | 92.0017 | 47.9491 | 76.8794 | 85.0907 | 70.7361 |
toos | 99.9076 | 93.1837 | 61.0307 | 71.7786 | 52.4901 |
MIREX 2009 Audio Melody Extraction Runtime Data
Participant | Machine | Runtime (dd:hh:mm) |
---|---|---|
CL1 | ALE | 00:00:28 |
CL2 | ALE | 00:00:33 |
DR1 | ALE | 16:00:00 |
DR2 | ALE | 00:08:44 |
HJC1 | FAST3 | 00:05:44 |
HJC2 | FAST3 | 00:09:38 |
JJY | ALE | 02:14:06 |
KD | ALE | 00:00:24 |
MW | ALE | 00:02:12 |
PC | BIGWIN | 03:05:57 |
RR | BIGWIN | 00:00:26 |
TOOS | ALE | 01:00:28 |