Difference between revisions of "2006:Audio Melody Extraction Results"
From MIREX Wiki
(→Team ID) |
|||
| (16 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
[[Category: Results]] | [[Category: Results]] | ||
==Introduction== | ==Introduction== | ||
| − | These are the results for the 2006 running of the Audio Melody Extraction task set. For background information about this task set please refer to the [[Audio Melody Extraction]] page. | + | These are the results for the 2006 running of the Audio Melody Extraction task set. For background information about this task set please refer to the [[2006:Audio Melody Extraction]] page. |
| + | |||
| + | The aim of the MIREX audio melody extraction evaluation is to identify the predominant melody pitch contour from polyphonic musical audio. The task consists of two parts: Voicing detection (deciding whether a particular time frame contains a "melody pitch" or not), and pitch detection (deciding the most likely melody pitch for each time frame). We structure the submission to allow these parts to be done independently, i.e. it is possible (via a negative pitch value) to guess a pitch even for frames that were being judged unvoiced. The algorithms were tested on two datasets, the MIREX2005 dataset, consisting of 25 sound files, and the ADC2004 dataset, consisting of 20 sound files, both across different music styles. Moreover, each of these datasets were split into two groups: those files in which the predominant melody is sung, and those in which the predominant melody is nonvocal. | ||
===General Legend=== | ===General Legend=== | ||
| − | ====Team ID==== | + | ====Team ID==== |
| − | '''dressler''' = [https://www.music-ir.org/ | + | '''dressler''' = [https://www.music-ir.org/mirex/abstracts/2006/AME_dressler.pdf Karin Dressler]<br /> |
| − | '''ryynanen''' = [https://www.music-ir.org/ | + | '''ryynanen''' = [https://www.music-ir.org/mirex/abstracts/2006/AME_ryynanen.pdf Matti Ryynänen and Anssi Klapuri]<br /> |
| − | '''poliner''' = Graham Poliner and | + | '''poliner''' = Graham Poliner and Daniel P. W. Ellis<br /> |
| − | '''sutton''' = [https://www.music-ir.org/ | + | '''sutton''' = [https://www.music-ir.org/mirex/abstracts/2006/AME_sutton.pdf Christopher Sutton, Emmanuel Vincent, Mark D. Plumbley and Juan P. Bello]<br /> |
| − | '''brossier''' = Paul Brossier<br /> | + | '''brossier''' = [https://www.music-ir.org/mirex/abstracts/2006/AME_BT_OD_TE_brossier.pdf Paul Brossier]<br /> |
| − | * Brossier does not do | + | * Brossier does not do voiced/unvoiced detection. |
| + | * Sutton's algorithm is designed for sung/vocal melody extraction. | ||
| − | + | ====Table Headings==== | |
| − | === | ||
'''Vx Recall''' = Voicing Detection<br /> | '''Vx Recall''' = Voicing Detection<br /> | ||
'''Vx False Alm''' = Voicing False Alarm<br /> | '''Vx False Alm''' = Voicing False Alarm<br /> | ||
| Line 23: | Line 25: | ||
'''Overall Acc''' = Overall Acuuracy<br /> | '''Overall Acc''' = Overall Acuuracy<br /> | ||
| + | ==Overall Summary Results== | ||
| − | == | + | ===MIREX 2006 Audio Melody Extraction Runtime Data=== |
| − | ===MIREX 2006 Audio Melody | + | <csv>2006/am06_runtime.csv</csv> |
| − | <csv>am06_adc04_all.csv</csv> | + | |
| + | ===MIREX 2006 Audio Melody Extraction Summary results - ADC 2004 Dataset - All=== | ||
| + | <csv>2006/am06_adc04_all.csv</csv> | ||
| − | [[Image: | + | [[Image:2006_am06_adc04_all.png]] |
Download the [https://www.music-ir.org/mirex2006/results/persong_adc04_all.xls Excel workbook] for ADC 2004 Dataset - All. | Download the [https://www.music-ir.org/mirex2006/results/persong_adc04_all.xls Excel workbook] for ADC 2004 Dataset - All. | ||
| − | ===MIREX 2006 Audio Melody | + | ===MIREX 2006 Audio Melody Extraction Summary results- ADC 2004 Dataset - Vocal=== |
| − | <csv>am06_adc04_vocal.csv</csv> | + | <csv>2006/am06_adc04_vocal.csv</csv> |
| − | [[Image: | + | [[Image:2006_am06_adc04_vocal.png]] |
Download the [https://www.music-ir.org/mirex2006/results/persong_adc04_vocal.xls Excel workbook] for ADC 2004 Dataset - Vocal. | Download the [https://www.music-ir.org/mirex2006/results/persong_adc04_vocal.xls Excel workbook] for ADC 2004 Dataset - Vocal. | ||
| − | ===MIREX 2006 Audio Melody | + | ===MIREX 2006 Audio Melody Extraction Summary results - ADC 2004 Dataset - Nonvocal=== |
| − | <csv>am06_adc04_nonvocal.csv</csv> | + | <csv>2006/am06_adc04_nonvocal.csv</csv> |
| − | [[Image: | + | [[Image:2006_am06 adc04 nonvocal.png]] |
Download the [https://www.music-ir.org/mirex2006/results/persong_adc04_nonvocal.xls Excel workbook] for ADC 2004 Dataset - Nonvocal. | Download the [https://www.music-ir.org/mirex2006/results/persong_adc04_nonvocal.xls Excel workbook] for ADC 2004 Dataset - Nonvocal. | ||
| − | ===MIREX 2006 Audio Melody | + | ===MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All=== |
| − | <csv>am06_m05_all.csv</csv> | + | <csv>2006/am06_m05_all.csv</csv> |
| − | [[Image: | + | [[Image:2006_am06 mirex05 all.png]] |
Download the [https://www.music-ir.org/mirex2006/results/persong_m05_all.xls Excel workbook] for MIREX 2005 Dataset - All. | Download the [https://www.music-ir.org/mirex2006/results/persong_m05_all.xls Excel workbook] for MIREX 2005 Dataset - All. | ||
| − | ===MIREX 2006 Audio Melody | + | ===MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - Vocal=== |
| − | <csv>am06_m05_vocal.csv</csv> | + | <csv>2006/am06_m05_vocal.csv</csv> |
| − | [[Image: | + | [[Image:2006_am06 mirex05 vocal.png]] |
Download the [https://www.music-ir.org/mirex2006/results/persong_m05_vocal.xls Excel workbook] for MIREX 2005 Dataset - Vocal. | Download the [https://www.music-ir.org/mirex2006/results/persong_m05_vocal.xls Excel workbook] for MIREX 2005 Dataset - Vocal. | ||
| − | ===MIREX 2006 Audio Melody | + | ===MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - Nonvocal=== |
| − | <csv>am06_m05_nonvocal.csv</csv> | + | <csv>2006/am06_m05_nonvocal.csv</csv> |
| − | [[Image: | + | [[Image:2006_am06 mirex05 nonvocal.png]] |
Download the [https://www.music-ir.org/mirex2006/results/persong_m05_nonvocal.xls Excel workbook] for MIREX 2005 Dataset - Nonvocal. | Download the [https://www.music-ir.org/mirex2006/results/persong_m05_nonvocal.xls Excel workbook] for MIREX 2005 Dataset - Nonvocal. | ||
Latest revision as of 10:49, 26 July 2010
Contents
- 1 Introduction
- 2 Overall Summary Results
- 2.1 MIREX 2006 Audio Melody Extraction Runtime Data
- 2.2 MIREX 2006 Audio Melody Extraction Summary results - ADC 2004 Dataset - All
- 2.3 MIREX 2006 Audio Melody Extraction Summary results- ADC 2004 Dataset - Vocal
- 2.4 MIREX 2006 Audio Melody Extraction Summary results - ADC 2004 Dataset - Nonvocal
- 2.5 MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All
- 2.6 MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - Vocal
- 2.7 MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - Nonvocal
Introduction
These are the results for the 2006 running of the Audio Melody Extraction task set. For background information about this task set please refer to the 2006:Audio Melody Extraction page.
The aim of the MIREX audio melody extraction evaluation is to identify the predominant melody pitch contour from polyphonic musical audio. The task consists of two parts: Voicing detection (deciding whether a particular time frame contains a "melody pitch" or not), and pitch detection (deciding the most likely melody pitch for each time frame). We structure the submission to allow these parts to be done independently, i.e. it is possible (via a negative pitch value) to guess a pitch even for frames that were being judged unvoiced. The algorithms were tested on two datasets, the MIREX2005 dataset, consisting of 25 sound files, and the ADC2004 dataset, consisting of 20 sound files, both across different music styles. Moreover, each of these datasets were split into two groups: those files in which the predominant melody is sung, and those in which the predominant melody is nonvocal.
General Legend
Team ID
dressler = Karin Dressler
ryynanen = Matti Ryynänen and Anssi Klapuri
poliner = Graham Poliner and Daniel P. W. Ellis
sutton = Christopher Sutton, Emmanuel Vincent, Mark D. Plumbley and Juan P. Bello
brossier = Paul Brossier
- Brossier does not do voiced/unvoiced detection.
- Sutton's algorithm is designed for sung/vocal melody extraction.
Table Headings
Vx Recall = Voicing Detection
Vx False Alm = Voicing False Alarm
Vx d' = Voicing d-prime
Raw pitch = Raw Pitch Accuracy
Raw Chroma = Raw Chroma Accuracy
Overall Acc = Overall Acuuracy
Overall Summary Results
MIREX 2006 Audio Melody Extraction Runtime Data
| Team ID | Data set | Machine | Run-time(seconds) |
|---|---|---|---|
| brossier | M05 | LIN | 58 |
| brossier | ADC04 | LIN | 30 |
| dressler | M05 | FAST | 48 |
| dressler | ADC04 | FAST | 27 |
| ryynanen | M05 | LIN/FAST | 773 |
| ryynanen | ADC04 | LIN/FAST | 440 |
| sutton | M05 | FAST | 8195 |
| sutton | ADC04 | FAST | 5014 |
MIREX 2006 Audio Melody Extraction Summary results - ADC 2004 Dataset - All
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| dressler | 90.9% | 10.5% | 2.58 | 82.9% | 84.0% | 82.5% |
| ryynanen | 84.4% | 12.6% | 2.16 | 80.6% | 82.3% | 77.3% |
| poliner | 89.9% | 36.3% | 1.63 | 73.2% | 76.4% | 71.9% |
| sutton | 73.2% | 24.9% | 1.30 | 62.6% | 65.4% | 58.2% |
| brossier | 99.7% | 88.4% | 1.61 | 57.4% | 68.7% | 49.6% |
Download the Excel workbook for ADC 2004 Dataset - All.
MIREX 2006 Audio Melody Extraction Summary results- ADC 2004 Dataset - Vocal
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| dressler | 89.8% | 10.9% | 2.50 | 77.1% | 78.0% | 77.3% |
| ryynanen | 85.9% | 11.5% | 2.28 | 78.3% | 79.3% | 76.2% |
| poliner | 88.4% | 34.5% | 1.59 | 65.4% | 69.0% | 64.7% |
| sutton | 90.8% | 32.0% | 1.79 | 67.5% | 68.0% | 64.2% |
| brossier | 99.8% | 93.9% | 1.28 | 56.3% | 63.5% | 46.7% |
Download the Excel workbook for ADC 2004 Dataset - Vocal.
MIREX 2006 Audio Melody Extraction Summary results - ADC 2004 Dataset - Nonvocal
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| dressler | 92.0% | 9.5% | 2.71 | 88.7% | 90.1% | 87.7% |
| ryynanen | 82.9% | 15.2% | 1.98 | 82.8% | 85.3% | 78.4% |
| poliner | 91.4% | 40.4% | 1.61 | 81.0% | 83.9% | 79.1% |
| sutton | 54.6% | 8.1% | 1.52 | 57.7% | 62.9% | 52.3% |
| brossier | 99.7% | 82.9% | 1.83 | 58.5% | 73.8% | 52.5% |
Download the Excel workbook for ADC 2004 Dataset - Nonvocal.
MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| dressler | 89.3% | 28.8% | 1.80 | 77.7% | 82.0% | 73.2% |
| ryynanen | 78.2% | 16.5% | 1.75 | 71.5% | 75.0% | 67.9% |
| poliner | 93.5% | 45.1% | 1.64 | 66.2% | 70.4% | 63.0% |
| sutton | 64.5% | 13.8% | 1.46 | 56.4% | 60.1% | 53.7% |
| brossier | 99.5% | 98.2% | 0.46 | 41.0% | 56.1% | 31.9% |
Download the Excel workbook for MIREX 2005 Dataset - All.
MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - Vocal
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| dressler | 85.5% | 28.7% | 1.62 | 78.5% | 81.6% | 73.7% |
| ryynanen | 77.0% | 15.6% | 1.75 | 75.7% | 76.9% | 72.5% |
| poliner | 93.7% | 44.3% | 1.68 | 69.1% | 70.6% | 65.0% |
| sutton | 71.8% | 12.3% | 1.74 | 70.7% | 71.6% | 67.3% |
| brossier | 99.6% | 97.9% | 0.63 | 42.7% | 53.5% | 30.7% |
Download the Excel workbook for MIREX 2005 Dataset - Vocal.
MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - Nonvocal
| Vx Recall | Vx False Alm | Vx d' | Raw pitch | Raw Chroma | Overall Acc | |
|---|---|---|---|---|---|---|
| dressler | 93.1% | 30.3% | 2.00 | 76.9% | 83.1% | 72.8% |
| ryynanen | 79.3% | 21.0% | 1.62 | 64.2% | 71.6% | 59.6% |
| poliner | 93.4% | 49.0% | 1.53 | 61.2% | 70.1% | 59.3% |
| sutton | 57.5% | 21.2% | 0.99 | 30.8% | 39.7% | 29.6% |
| brossier | 99.2% | 98.8% | 0.16 | 37.8% | 60.8% | 34.1% |
Download the Excel workbook for MIREX 2005 Dataset - Nonvocal.
