2008:Audio Melody Extraction Results

Introduction

These are the results for the 2008 running of the Audio Melody Extraction task set. For background information about this task set please refer to the 2008:Audio Melody Extraction page. Special thanks to Jean-Louis Durrieu for doing the vocal/non-vocal split summaries.

General Legend

Team ID

PC = P. Cancela
CLLY1 = C. Cao, M. Li, J. Liu, Y. Yan 1
CLLY2 = C. Cao, M. Li, J. Liu, Y. Yan 2
DRD1 = J-L. Durrieu, G. Richard, B. David 1
DRD2 = J-L. Durrieu, G. Richard, B. David 2
RK = M. Ryynänen, A. Klapuri
VR = V. Rao, P. Rao

Table Headings

Vx Recall = Voicing Detection
Vx False Alm = Voicing False Alarm
Vx d' = Voicing d-prime
Raw pitch = Raw Pitch Accuracy
Raw Chroma = Raw Chroma Accuracy
Overall Acc = Overall Acuuracy

Overall Summary Results

MIREX 2008 Audio Melody Extraction Overall Summary results - Weighted (by Number of Files) Avg. of all Datasets - All

	Avg. Overall Acc.
clly1	49.8%
clly2	62.1%
drd1	58.6%
drd2	73.2%
pc	76.1%
rk	71.1%
vr	67.1%

download these results as csv

MIREX 2008 Audio Melody Extraction Summary results - MIREX 2008 Dataset - All

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
drd1	98.7%	68.9%	1.74	85.8%	88.3%	76.0%
rk	90.4%	48.5%	1.34	83.5%	83.8%	75.3%
drd2	96.6%	57.1%	1.65	81.8%	82.6%	75.0%
pc	95.7%	65.4%	1.32	83.9%	84.0%	73.3%
vr	68.3%	21.1%	1.28	88.2%	88.6%	66.7%
clly1	77.6%	43.4%	0.92	54.7%	55.3%	51.4%
clly2	86.1%	67.8%	0.62	54.7%	55.3%	49.7%

download these results as csv

Download the Excel workbook for MIREX 2008 Dataset - All (NB: it seems that all the songs of this dataset are Vocal ones, there is therefore no vocal/non-vocal separate results).

MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - vocal

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
pc	83.1%	19.7%	1.81	77.6%	78.1%	76.8%
vr	81.5%	20.5%	1.72	76.1%	79.2%	71.8%
rk	89.5%	45.0%	1.38	74.6%	76.6%	67.6%
drd2	92.6%	51.1%	1.42	75.0%	77.9%	66.7%
clly2	81.5%	46.0%	1.00	70.1%	72.8%	63.7%
clly1	75.1%	37.5%	1.00	70.1%	72.8%	62.7%
drd1	91.7%	63.6%	1.04	54.7%	62.4%	48.0%

download these results as csv

Download the Excel Workbook for MIREX 2005 Dataset - vocal.

MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - nonvocal

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
drd2	90.4%	39.7%	1.56	67.2%	72.6%	64.1%
drd1	88.1%	44.4%	1.32	60.1%	68.3%	57.7%
pc	80.9%	21.1%	1.68	58.3%	62.4%	56.5%
clly2	71.1%	50.0%	0.56	64.6%	68.3%	55.1%
rk	78.4%	41.5%	1.00	63.4%	68.7%	54.8%
vr	84.7%	40.7%	1.26	56.9%	69.9%	51.5%
clly1	27.8%	50.5%	-0.60	64.6%	68.3%	24.1%

download these results as csv

Download the Excel Workbook for MIREX 2005 Dataset - nonvocal.

MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
pc	83.9%	19.9%	1.83	71.0%	72.8%	69.8%
drd2	92.0%	49.2%	1.42	72.4%	76.2%	66.0%
vr	85.1%	23.8%	1.75	69.7%	76.5%	64.9%
rk	85.6%	44.4%	1.21	71.2%	74.4%	63.5%
clly2	77.8%	46.7%	0.85	68.9%	72.0%	61.4%
drd1	91.9%	60.4%	1.14	57.4%	65.3%	52.2%
clly1	58.2%	42.2%	0.40	68.9%	72.0%	48.9%

download these results as csv

Download the Excel Workbook for MIREX 2005 Dataset - All.

MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - vocal

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
pc	91.4%	11.0%	2.60	89.5%	89.8%	88.3%
drd2	92.7%	30.9%	1.95	87.4%	87.6%	83.2%
rk	89.2%	29.3%	1.78	84.0%	84.2%	79.9%
vr	80.2%	20.9%	1.66	81.6%	88.1%	72.0%
drd1	95.4%	59.8%	1.44	77.2%	83.4%	69.6%
clly2	87.3%	53.1%	1.06	71.8%	72.3%	64.9%
clly1	62.5%	32.2%	0.78	71.8%	72.3%	52.3%

download these results as csv

Download the Excel Workbook for ADC 2004 Dataset - vocal.

MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - nonvocal

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
pc	82.6%	6.9%	2.42	78.5%	80.1%	80.2%
drd2	90.2%	33.2%	1.72	83.3%	84.1%	79.0%
rk	88.4%	18.9%	2.07	80.2%	82.6%	77.2%
clly2	80.2%	39.7%	1.11	80.4%	83.4%	72.7%
vr	85.7%	17.7%	1.99	70.4%	80.8%	67.2%
clly1	45.4%	26.1%	0.53	80.4%	83.4%	47.2%
drd1	91.3%	53.5%	1.27	49.0%	62.5%	44.6%

download these results as csv

Download the Excel Workbook for ADC 2004 Dataset - nonvocal.

MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - All

	Vx Recall	Vx False Alm	Vx d'	Raw pitch	Raw Chroma	Overall Acc
pc	88.0%	9.8%	2.47	85.1%	85.9%	85.1%
drd2	91.7%	31.6%	1.86	85.7%	86.2%	81.5%
rk	88.9%	26.2%	1.86	82.4%	83.5%	78.8%
vr	82.3%	20.0%	1.77	77.1%	85.2%	70.1%
clly2	84.6%	49.1%	1.04	75.3%	76.7%	68.0%
drd1	93.8%	57.9%	1.34	65.9%	75.0%	59.6%
clly1	55.7%	29.7%	0.67	75.3%	76.7%	50.2%

download these results as csv

Download the Excel workbook for ADC 2004 Dataset - All.

MIREX 2008 Audio Melody Extraction Runtime Data

Team ID	Data set	Machine	Run-time(seconds)
clly1	M05	FAST3	56
clly1	ADC04	FAST3	36
clly1	M08	FAST3	49
clly2	M05	FAST3	57
clly2	ADC04	FAST3	39
clly2	M08	FAST3	50
drd1	M05	ALE	3 days
drd1	ADC04	ALE	5 days
drd1	M08	ALE	4 days
drd2	M05	ALE	3294
drd2	ADC04	ALE	2082
drd2	M08	ALE	2318
pc	M05	FAST	61470
pc	ADC04	FAST	33974
pc	M08	FAST	47031
rk	M05	ALE	115
rk	ADC04	ALE	73
rk	M08	ALE	100
vr	M05	FAST	200
vr	ADC04	FAST	110
vr	M08	FAST	184

download these results as csv