Difference between revisions of "2007:Query-by-Singing/Humming Results"
CameronJones (talk | contribs) (→Results Summaries) |
(→Task 2) |
||
| Line 27: | Line 27: | ||
====Task 2==== | ====Task 2==== | ||
| + | |||
| + | ===General Legend=== | ||
| + | |||
| + | CG_XW FH_XW NM_XW RJ1_RJ RJ1_XW RJ2_RJ RJ2_XW XW1_XW XW2_XW XW2_RJ | ||
| + | ====Team ID==== | ||
| + | '''FH_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_ferraro.pdf Pascal Ferraro, Pierre Hanna, Julien Allali, Matthias Robine based on XW note transcriber]<br /> | ||
| + | '''CG_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_SMS_gomez.pdf Carlos G├│mez, Soraya Abad-Mota, Edna Ruckhaus based on XW note transcriber]<br /> | ||
| + | '''RJ1_RJ''' = [https://www.music-ir.org/mirex2007/abs/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1 based on RJ pitch transcriber]<br /> | ||
| + | '''RJ1_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1 based on XW pitch transcriber]<br /> | ||
| + | '''RJ2_RJ''' = [https://www.music-ir.org/mirex2007/abs/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2 based on RJ pitch transcriber]<br /> | ||
| + | '''RJ2_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_jang.pdf J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2 based on XW pitch transcriber]<br /> | ||
| + | '''NM''' = [https://www.music-ir.org/mirex2007/abs/QBSH_lemstrom.pdf Kjell Lemström, Niko Mikkilä based on XW note transcriber]<br /> | ||
| + | '''XW1_XW''' = [https://www.music-ir.org/mirex2007/abs/QBSH_wu.pdf Xiao Wu, Ming Li 1 based on XW note transcriber]<br /> | ||
| + | '''XW2''' = [https://www.music-ir.org/mirex2007/abs/QBSH_wu.pdf Xiao Wu, Ming Li 2 based on XW pitch transcriber]<br /> | ||
<csv>qbsh07_task_2_overall.csv</csv> | <csv>qbsh07_task_2_overall.csv</csv> | ||
Revision as of 21:20, 16 September 2007
Contents
Introduction
These are the results for the 2007 running of the Query-by-Singing/Humming task. For background information about this task set please refer to the Query by Singing/Humming page. The first subtask is the same as last years. In this subtask, submitted systems take a sung query as input and return a list of songs from the test database. Mean reciprocal rank (MRR) of the ground truth is calculated over the top 20 returns. The test database consists of 48 ground-truth MIDIs + 2000 Essen Collection MIDI noise files. The query database consists of 2797 sung queries. In the second subtask, the same setup as the first subtask used with combination of different transcribers and matchers.
General Legend
Team ID
FH = Pascal Ferraro, Pierre Hanna, Julien Allali, Matthias Robine
CG = Carlos G├│mez, Soraya Abad-Mota, Edna Ruckhaus
RJ1 = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1
RJ2 = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2
NM = Kjell Lemström, Niko Mikkilä
XW1 = Xiao Wu, Ming Li 1
XW2 = Xiao Wu, Ming Li 2
AU1 = Alexandra L. Uitdenbogerd 1
AU2 = Alexandra L. Uitdenbogerd 2
AU3 = Alexandra L. Uitdenbogerd 3
Results Summaries
Task 1
The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05); file /nema-raid/www/mirex/results/qbsh07_task1_sum_friedmans.csv not found file /nema-raid/www/mirex/results/qbsh07_task1_detail_friedmans.csv not found
Task 1 Summary Results by Query Group
file /nema-raid/www/mirex/results/qbsh07_task1_avg_per_group.csv not found
Task 2
General Legend
CG_XW FH_XW NM_XW RJ1_RJ RJ1_XW RJ2_RJ RJ2_XW XW1_XW XW2_XW XW2_RJ
Team ID
FH_XW = Pascal Ferraro, Pierre Hanna, Julien Allali, Matthias Robine based on XW note transcriber
CG_XW = Carlos G├│mez, Soraya Abad-Mota, Edna Ruckhaus based on XW note transcriber
RJ1_RJ = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1 based on RJ pitch transcriber
RJ1_XW = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 1 based on XW pitch transcriber
RJ2_RJ = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2 based on RJ pitch transcriber
RJ2_XW = J.-S. Roger Jang, Nien-Jung Lee, Chao-Ling Hsu 2 based on XW pitch transcriber
NM = Kjell Lemström, Niko Mikkilä based on XW note transcriber
XW1_XW = Xiao Wu, Ming Li 1 based on XW note transcriber
XW2 = Xiao Wu, Ming Li 2 based on XW pitch transcriber
file /nema-raid/www/mirex/results/qbsh07_task_2_overall.csv not found
Task 2 Friedman's Test for Significant Differences
The Friedman test was run in MATLAB against the QBSH Task 1 MRR data over the 48 ground truth song groups. Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05); file /nema-raid/www/mirex/results/qbsh07_task2_sum_friedmans.csv not found file /nema-raid/www/mirex/results/qbsh07_task2_detail_friedmans.csv not found
Task 2 Summary Results by Query Group
file /nema-raid/www/mirex/results/qbsh07_task2_avg_per_group.csv not found
