Difference between revisions of "2008:Audio Melody Extraction Results"
From MIREX Wiki
IMIRSELBot (talk | contribs) m (Robot: Automated text replacement (-\[http://www.music-ir.org/mirex/(200[5-9])/abs/ +[http://www.music-ir.org/mirex/abs/\1/)) |
IMIRSELBot (talk | contribs) m (Robot: Automated text replacement (-\[\[Image:\s*([A-Z]) +[[Image:2008_\1)) |
||
| Line 29: | Line 29: | ||
<csv>2008/am08_m08_all.csv</csv> | <csv>2008/am08_m08_all.csv</csv> | ||
| − | [[Image: | + | [[Image:2008_Am08_m08_all.png]] |
Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_m08_all.xls Excel workbook] for MIREX 2008 Dataset - All (NB: it seems that all the songs of this dataset are Vocal ones, there is therefore no vocal/non-vocal separate results). | Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_m08_all.xls Excel workbook] for MIREX 2008 Dataset - All (NB: it seems that all the songs of this dataset are Vocal ones, there is therefore no vocal/non-vocal separate results). | ||
| Line 36: | Line 36: | ||
<csv>2008/am08_m05_vocal.csv</csv> | <csv>2008/am08_m05_vocal.csv</csv> | ||
| − | [[Image: | + | [[Image:2008_Am08_m05_vocal.png]] |
Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_m05_vocal.xls Excel Workbook] for MIREX 2005 Dataset - vocal. | Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_m05_vocal.xls Excel Workbook] for MIREX 2005 Dataset - vocal. | ||
| Line 43: | Line 43: | ||
<csv>2008/am08_m05_nonvocal.csv</csv> | <csv>2008/am08_m05_nonvocal.csv</csv> | ||
| − | [[Image: | + | [[Image:2008_Am08_m05_nonvocal.png]] |
Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_m05_nonvocal.xls Excel Workbook] for MIREX 2005 Dataset - nonvocal. | Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_m05_nonvocal.xls Excel Workbook] for MIREX 2005 Dataset - nonvocal. | ||
| Line 50: | Line 50: | ||
<csv>2008/am08_m05_all.csv</csv> | <csv>2008/am08_m05_all.csv</csv> | ||
| − | [[Image: | + | [[Image:2008_Am08_m05_all2.png]] |
Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_m05_all.xls Excel Workbook] for MIREX 2005 Dataset - All. | Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_m05_all.xls Excel Workbook] for MIREX 2005 Dataset - All. | ||
| Line 57: | Line 57: | ||
<csv>2008/am08_adc04_vocal.csv</csv> | <csv>2008/am08_adc04_vocal.csv</csv> | ||
| − | [[Image: | + | [[Image:2008_Am08_adc04_vocal.png]] |
Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_adc04_vocal.xls Excel Workbook] for ADC 2004 Dataset - vocal. | Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_adc04_vocal.xls Excel Workbook] for ADC 2004 Dataset - vocal. | ||
| Line 64: | Line 64: | ||
<csv>2008/am08_adc04_nonvocal.csv</csv> | <csv>2008/am08_adc04_nonvocal.csv</csv> | ||
| − | [[Image: | + | [[Image:2008_Am08_adc04_nonvocal.png]] |
Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_adc04_nonvocal.xls Excel Workbook] for ADC 2004 Dataset - nonvocal. | Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_adc04_nonvocal.xls Excel Workbook] for ADC 2004 Dataset - nonvocal. | ||
| Line 71: | Line 71: | ||
<csv>2008/am08_adc04_all.csv</csv> | <csv>2008/am08_adc04_all.csv</csv> | ||
| − | [[Image: | + | [[Image:2008_Am08_adc04_all.png]] |
Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_adc04_all.xls Excel workbook] for ADC 2004 Dataset - All. | Download the [https://www.music-ir.org/mirex/2008/results/am08_persong_adc04_all.xls Excel workbook] for ADC 2004 Dataset - All. | ||
Revision as of 21:32, 13 May 2010
Contents
- 1 Introduction
- 2 Overall Summary Results
- 2.1 MIREX 2008 Audio Melody Extraction Overall Summary results - Weighted (by Number of Files) Avg. of all Datasets - All
- 2.2 MIREX 2008 Audio Melody Extraction Summary results - MIREX 2008 Dataset - All
- 2.3 MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - vocal
- 2.4 MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - nonvocal
- 2.5 MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All
- 2.6 MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - vocal
- 2.7 MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - nonvocal
- 2.8 MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - All
- 2.9 MIREX 2008 Audio Melody Extraction Runtime Data
Introduction
These are the results for the 2008 running of the Audio Melody Extraction task set. For background information about this task set please refer to the 2008:Audio Melody Extraction page. Special thanks to Jean-Louis Durrieu for doing the vocal/non-vocal split summaries.
General Legend
Team ID
PC = P. Cancela
CLLY1 = C. Cao, M. Li, J. Liu, Y. Yan 1
CLLY2 = C. Cao, M. Li, J. Liu, Y. Yan 2
DRD1 = J-L. Durrieu, G. Richard, B. David 1
DRD2 = J-L. Durrieu, G. Richard, B. David 2
RK = M. Ryynänen, A. Klapuri
VR = V. Rao, P. Rao
Table Headings
Vx Recall = Voicing Detection
Vx False Alm = Voicing False Alarm
Vx d' = Voicing d-prime
Raw pitch = Raw Pitch Accuracy
Raw Chroma = Raw Chroma Accuracy
Overall Acc = Overall Acuuracy
Overall Summary Results
MIREX 2008 Audio Melody Extraction Overall Summary results - Weighted (by Number of Files) Avg. of all Datasets - All
| Avg. Overall Acc. | |
|---|---|
| clly1 | 49.8% |
| clly2 | 62.1% |
| drd1 | 58.6% |
| drd2 | 73.2% |
| pc | 76.1% |
| rk | 71.1% |
| vr | 67.1% |
MIREX 2008 Audio Melody Extraction Summary results - MIREX 2008 Dataset - All
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| drd1 | 98.7% | 68.9% | 1.74 | 85.8% | 88.3% | 76.0% |
| rk | 90.4% | 48.5% | 1.34 | 83.5% | 83.8% | 75.3% |
| drd2 | 96.6% | 57.1% | 1.65 | 81.8% | 82.6% | 75.0% |
| pc | 95.7% | 65.4% | 1.32 | 83.9% | 84.0% | 73.3% |
| vr | 68.3% | 21.1% | 1.28 | 88.2% | 88.6% | 66.7% |
| clly1 | 77.6% | 43.4% | 0.92 | 54.7% | 55.3% | 51.4% |
| clly2 | 86.1% | 67.8% | 0.62 | 54.7% | 55.3% | 49.7% |
Download the Excel workbook for MIREX 2008 Dataset - All (NB: it seems that all the songs of this dataset are Vocal ones, there is therefore no vocal/non-vocal separate results).
MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - vocal
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| pc | 83.1% | 19.7% | 1.81 | 77.6% | 78.1% | 76.8% |
| vr | 81.5% | 20.5% | 1.72 | 76.1% | 79.2% | 71.8% |
| rk | 89.5% | 45.0% | 1.38 | 74.6% | 76.6% | 67.6% |
| drd2 | 92.6% | 51.1% | 1.42 | 75.0% | 77.9% | 66.7% |
| clly2 | 81.5% | 46.0% | 1.00 | 70.1% | 72.8% | 63.7% |
| clly1 | 75.1% | 37.5% | 1.00 | 70.1% | 72.8% | 62.7% |
| drd1 | 91.7% | 63.6% | 1.04 | 54.7% | 62.4% | 48.0% |
Download the Excel Workbook for MIREX 2005 Dataset - vocal.
MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - nonvocal
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| drd2 | 90.4% | 39.7% | 1.56 | 67.2% | 72.6% | 64.1% |
| drd1 | 88.1% | 44.4% | 1.32 | 60.1% | 68.3% | 57.7% |
| pc | 80.9% | 21.1% | 1.68 | 58.3% | 62.4% | 56.5% |
| clly2 | 71.1% | 50.0% | 0.56 | 64.6% | 68.3% | 55.1% |
| rk | 78.4% | 41.5% | 1.00 | 63.4% | 68.7% | 54.8% |
| vr | 84.7% | 40.7% | 1.26 | 56.9% | 69.9% | 51.5% |
| clly1 | 27.8% | 50.5% | -0.60 | 64.6% | 68.3% | 24.1% |
File:2008 Am08 m05 nonvocal.png
Download the Excel Workbook for MIREX 2005 Dataset - nonvocal.
MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| pc | 83.9% | 19.9% | 1.83 | 71.0% | 72.8% | 69.8% |
| drd2 | 92.0% | 49.2% | 1.42 | 72.4% | 76.2% | 66.0% |
| vr | 85.1% | 23.8% | 1.75 | 69.7% | 76.5% | 64.9% |
| rk | 85.6% | 44.4% | 1.21 | 71.2% | 74.4% | 63.5% |
| clly2 | 77.8% | 46.7% | 0.85 | 68.9% | 72.0% | 61.4% |
| drd1 | 91.9% | 60.4% | 1.14 | 57.4% | 65.3% | 52.2% |
| clly1 | 58.2% | 42.2% | 0.40 | 68.9% | 72.0% | 48.9% |
Download the Excel Workbook for MIREX 2005 Dataset - All.
MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - vocal
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| pc | 91.4% | 11.0% | 2.60 | 89.5% | 89.8% | 88.3% |
| drd2 | 92.7% | 30.9% | 1.95 | 87.4% | 87.6% | 83.2% |
| rk | 89.2% | 29.3% | 1.78 | 84.0% | 84.2% | 79.9% |
| vr | 80.2% | 20.9% | 1.66 | 81.6% | 88.1% | 72.0% |
| drd1 | 95.4% | 59.8% | 1.44 | 77.2% | 83.4% | 69.6% |
| clly2 | 87.3% | 53.1% | 1.06 | 71.8% | 72.3% | 64.9% |
| clly1 | 62.5% | 32.2% | 0.78 | 71.8% | 72.3% | 52.3% |
File:2008 Am08 adc04 vocal.png
Download the Excel Workbook for ADC 2004 Dataset - vocal.
MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - nonvocal
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| pc | 82.6% | 6.9% | 2.42 | 78.5% | 80.1% | 80.2% |
| drd2 | 90.2% | 33.2% | 1.72 | 83.3% | 84.1% | 79.0% |
| rk | 88.4% | 18.9% | 2.07 | 80.2% | 82.6% | 77.2% |
| clly2 | 80.2% | 39.7% | 1.11 | 80.4% | 83.4% | 72.7% |
| vr | 85.7% | 17.7% | 1.99 | 70.4% | 80.8% | 67.2% |
| clly1 | 45.4% | 26.1% | 0.53 | 80.4% | 83.4% | 47.2% |
| drd1 | 91.3% | 53.5% | 1.27 | 49.0% | 62.5% | 44.6% |
File:2008 Am08 adc04 nonvocal.png
Download the Excel Workbook for ADC 2004 Dataset - nonvocal.
MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - All
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| pc | 88.0% | 9.8% | 2.47 | 85.1% | 85.9% | 85.1% |
| drd2 | 91.7% | 31.6% | 1.86 | 85.7% | 86.2% | 81.5% |
| rk | 88.9% | 26.2% | 1.86 | 82.4% | 83.5% | 78.8% |
| vr | 82.3% | 20.0% | 1.77 | 77.1% | 85.2% | 70.1% |
| clly2 | 84.6% | 49.1% | 1.04 | 75.3% | 76.7% | 68.0% |
| drd1 | 93.8% | 57.9% | 1.34 | 65.9% | 75.0% | 59.6% |
| clly1 | 55.7% | 29.7% | 0.67 | 75.3% | 76.7% | 50.2% |
Download the Excel workbook for ADC 2004 Dataset - All.
MIREX 2008 Audio Melody Extraction Runtime Data
| Team ID | Data set | Machine | Run-time(seconds) |
|---|---|---|---|
| clly1 | M05 | FAST3 | 56 |
| clly1 | ADC04 | FAST3 | 36 |
| clly1 | M08 | FAST3 | 49 |
| clly2 | M05 | FAST3 | 57 |
| clly2 | ADC04 | FAST3 | 39 |
| clly2 | M08 | FAST3 | 50 |
| drd1 | M05 | ALE | 3 days |
| drd1 | ADC04 | ALE | 5 days |
| drd1 | M08 | ALE | 4 days |
| drd2 | M05 | ALE | 3294 |
| drd2 | ADC04 | ALE | 2082 |
| drd2 | M08 | ALE | 2318 |
| pc | M05 | FAST | 61470 |
| pc | ADC04 | FAST | 33974 |
| pc | M08 | FAST | 47031 |
| rk | M05 | ALE | 115 |
| rk | ADC04 | ALE | 73 |
| rk | M08 | ALE | 100 |
| vr | M05 | FAST | 200 |
| vr | ADC04 | FAST | 110 |
| vr | M08 | FAST | 184 |
