2006:QBSH: Query-by-Singing/Humming Results

Introduction

These are the results for the 2006 running of the QBSH: Query-by-Singing Humming task set. For background information about this task set please refer to the QBSH: Query-by-Singing/Humming page. QBSH task consists of two sub tasks. First one is Known-Item Retrieval. The system takes a sung query as input and returns a list of songs from the test database. Mean reciprocal rank of the ground truth is calculated over the top 20 returns. The test database consists of 48 ground truth and 2000 noise midi files. The query database consists of 2797 sung queries.

The second task is the Variants Retrieval. The System takes an input from the query database which consists of 2797 sung queries + 48 ground truth files and returns a list of 20 songs. The precision is calculated over the returns for each query.

General Legend

Team ID

AU = Alexandra Uitdenbogerd
CS1 = Christian Sailer-ear
CS2 = Christian Sailer-midi
CS3 = Christian Sailer-warp
FH = Pascal Ferraro and Pierre Hanna
NM = Kjell Lemström, Niko Mikkilä, Veli Mäkinen and Esko Ukkonen
RJ = Jyh-Shing Roger Jang and Nien-Jung Lee
RL = Ernesto Lopez and Martín Rocamora
RT = Rainer Typke, Frans Wiering and Remco C. Veltkamp
XW = Xiao Wu and Ming Li

Calculating Summary Measures

MRR = Mean Reciprocal Rank. Reciprocal rank is the reciprocal of the rank of the first correctly identified cover for each query (1/rank). These values are averaged for each ground truth group as well as overall.

Overall Summary Results

file /nema-raid/www/mirex/results/qbsh_overall_sum.csv not found

QBSH Task 1 Runtime Data

file /nema-raid/www/mirex/results/qbsh06_task1_runtime.csv not found

QBSH Task 2 Runtime Data

file /nema-raid/www/mirex/results/qbsh06_task2_runtime.csv not found

Task I: "Known-Item Searching" Friedman Test with Multiple Comparisons Results (p=0.05)

The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups.
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05); file /nema-raid/www/mirex/results/qush_friedman.csv not found file /nema-raid/www/mirex/results/QBSH_sum.csv not found

Task I: "Known-Item Searching" Summary Results

MMR data summarized by the 48 ground truth song groups. file /nema-raid/www/mirex/results/qbsh_task1_sum.csv not found

Task II: "Queries as Variations" Friedman Test with Multiple Comparisons Results (p=0.05)

The Friedman test was run in MATLAB against the QBSH Task 2 mean precision data over the 48 ground truth song groups.
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05); file /nema-raid/www/mirex/results/qbsh2_friedman.csv not found file /nema-raid/www/mirex/results/qbsh2_sum.csv not found

Task II: "Queries as Variations" Summary Results

Mean precision data summarized over the 48 ground truth song groups. file /nema-raid/www/mirex/results/qbsh06_task2_sum.csv not found

Raw Scores

The raw data are located on the Query-by-Singing/Humming Raw Data page.