Difference between revisions of "2006:QBSH: Query-by-Singing/Humming"
| Line 1: | Line 1: | ||
==Overview== | ==Overview== | ||
| + | The QBSH: Query-by-Singing/Humming is a new task for MIREX 2006. This goal of the QBSH task is the evaluation of MIR systems that take as query input queries sung or hummed by real-world users. Prof. J.-S. Roger Jang has created a set of 2719 recorded query files representing requests for 48 different music pieces. | ||
| − | ==Task | + | ==Task Descriptions== |
| + | 1. There will be *TWO* QBSH tasks (thus TWO *index* buildings and TWO *test* runs): | ||
| + | |||
| + | TASK1: Based on Prof. Jang's original idea. | ||
| + | |||
| + | --Test database: 48 ground truth MIDIs + ~2000 Essen MIDI noise files | ||
| + | --Queries: 2719 sung queries [1] | ||
| + | --Evaluation: Mean Reciprocal rank over top X returns | ||
| + | |||
| + | |||
| + | TASK2: Based on Stephen's idea that queries are variants of ground truth | ||
| + | |||
| + | --Test database--48 ground truth MIDIs + ~ 2000 Essen MIDI noise files + 2719 sung queries [1] | ||
| + | --Queries: 2719 sung queries + 48 ground truth MIDI files | ||
| + | --Evaluation: Classic precision and recall over X top returns | ||
| + | |||
| + | |||
| + | ==Moderators== | ||
| + | J.-S. Roger Jang | ||
| + | J. Stephen Downie | ||
| + | Rainer Typke | ||
Revision as of 08:38, 21 July 2006
Overview
The QBSH: Query-by-Singing/Humming is a new task for MIREX 2006. This goal of the QBSH task is the evaluation of MIR systems that take as query input queries sung or hummed by real-world users. Prof. J.-S. Roger Jang has created a set of 2719 recorded query files representing requests for 48 different music pieces.
Task Descriptions
1. There will be *TWO* QBSH tasks (thus TWO *index* buildings and TWO *test* runs):
TASK1: Based on Prof. Jang's original idea.
--Test database: 48 ground truth MIDIs + ~2000 Essen MIDI noise files --Queries: 2719 sung queries [1] --Evaluation: Mean Reciprocal rank over top X returns
TASK2: Based on Stephen's idea that queries are variants of ground truth
--Test database--48 ground truth MIDIs + ~ 2000 Essen MIDI noise files + 2719 sung queries [1] --Queries: 2719 sung queries + 48 ground truth MIDI files --Evaluation: Classic precision and recall over X top returns
Moderators
J.-S. Roger Jang J. Stephen Downie Rainer Typke
