2006:Audio Melody Extraction Results

Introduction

These are the results for the 2006 running of the Audio Melody Extraction task set. For background information about this task set please refer to the 2006:Audio Melody Extraction page.

The aim of the MIREX audio melody extraction evaluation is to identify the predominant melody pitch contour from polyphonic musical audio. The task consists of two parts: Voicing detection (deciding whether a particular time frame contains a "melody pitch" or not), and pitch detection (deciding the most likely melody pitch for each time frame). We structure the submission to allow these parts to be done independently, i.e. it is possible (via a negative pitch value) to guess a pitch even for frames that were being judged unvoiced. The algorithms were tested on two datasets, the MIREX2005 dataset, consisting of 25 sound files, and the ADC2004 dataset, consisting of 20 sound files, both across different music styles. Moreover, each of these datasets were split into two groups: those files in which the predominant melody is sung, and those in which the predominant melody is nonvocal.

General Legend

Team ID

dressler = Karin Dressler
ryynanen = Matti Ryynänen and Anssi Klapuri
poliner = Graham Poliner and Daniel P. W. Ellis
sutton = Christopher Sutton, Emmanuel Vincent, Mark D. Plumbley and Juan P. Bello
brossier = Paul Brossier

• Brossier does not do voiced/unvoiced detection.
• Sutton's algorithm is designed for sung/vocal melody extraction.

Table Headings

Vx Recall = Voicing Detection
Vx False Alm = Voicing False Alarm
Vx d' = Voicing d-prime
Raw pitch = Raw Pitch Accuracy
Raw Chroma = Raw Chroma Accuracy
Overall Acc = Overall Acuuracy

Overall Summary Results

MIREX 2006 Audio Melody Extraction Runtime Data

Team ID	Data set	Machine	Run-time(seconds)
brossier	M05	LIN	58
brossier	ADC04	LIN	30
dressler	M05	FAST	48
dressler	ADC04	FAST	27
ryynanen	M05	LIN/FAST	773
ryynanen	ADC04	LIN/FAST	440
sutton	M05	FAST	8195
sutton	ADC04	FAST	5014

download these results as csv

MIREX 2006 Audio Melody Extraction Summary results - ADC 2004 Dataset - All

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
dressler	90.9%	10.5%	2.58	82.9%	84.0%	82.5%
ryynanen	84.4%	12.6%	2.16	80.6%	82.3%	77.3%
poliner	89.9%	36.3%	1.63	73.2%	76.4%	71.9%
sutton	73.2%	24.9%	1.30	62.6%	65.4%	58.2%
brossier	99.7%	88.4%	1.61	57.4%	68.7%	49.6%

download these results as csv

Download the Excel workbook for ADC 2004 Dataset - All.

MIREX 2006 Audio Melody Extraction Summary results- ADC 2004 Dataset - Vocal

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
dressler	89.8%	10.9%	2.50	77.1%	78.0%	77.3%
ryynanen	85.9%	11.5%	2.28	78.3%	79.3%	76.2%
poliner	88.4%	34.5%	1.59	65.4%	69.0%	64.7%
sutton	90.8%	32.0%	1.79	67.5%	68.0%	64.2%
brossier	99.8%	93.9%	1.28	56.3%	63.5%	46.7%

download these results as csv

Download the Excel workbook for ADC 2004 Dataset - Vocal.

MIREX 2006 Audio Melody Extraction Summary results - ADC 2004 Dataset - Nonvocal

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
dressler	92.0%	9.5%	2.71	88.7%	90.1%	87.7%
ryynanen	82.9%	15.2%	1.98	82.8%	85.3%	78.4%
poliner	91.4%	40.4%	1.61	81.0%	83.9%	79.1%
sutton	54.6%	8.1%	1.52	57.7%	62.9%	52.3%
brossier	99.7%	82.9%	1.83	58.5%	73.8%	52.5%

download these results as csv

Download the Excel workbook for ADC 2004 Dataset - Nonvocal.

MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
dressler	89.3%	28.8%	1.80	77.7%	82.0%	73.2%
ryynanen	78.2%	16.5%	1.75	71.5%	75.0%	67.9%
poliner	93.5%	45.1%	1.64	66.2%	70.4%	63.0%
sutton	64.5%	13.8%	1.46	56.4%	60.1%	53.7%
brossier	99.5%	98.2%	0.46	41.0%	56.1%	31.9%

download these results as csv

Download the Excel workbook for MIREX 2005 Dataset - All.

MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - Vocal

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
dressler	85.5%	28.7%	1.62	78.5%	81.6%	73.7%
ryynanen	77.0%	15.6%	1.75	75.7%	76.9%	72.5%
poliner	93.7%	44.3%	1.68	69.1%	70.6%	65.0%
sutton	71.8%	12.3%	1.74	70.7%	71.6%	67.3%
brossier	99.6%	97.9%	0.63	42.7%	53.5%	30.7%

download these results as csv

Download the Excel workbook for MIREX 2005 Dataset - Vocal.

MIREX 2006 Audio Melody Extraction Summary results - MIREX 2005 Dataset - Nonvocal

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
dressler	93.1%	30.3%	2.00	76.9%	83.1%	72.8%
ryynanen	79.3%	21.0%	1.62	64.2%	71.6%	59.6%
poliner	93.4%	49.0%	1.53	61.2%	70.1%	59.3%
sutton	57.5%	21.2%	0.99	30.8%	39.7%	29.6%
brossier	99.2%	98.8%	0.16	37.8%	60.8%	34.1%

download these results as csv

Download the Excel workbook for MIREX 2005 Dataset - Nonvocal.