Difference between revisions of "2006:QBSH: Query-by-Singing/Humming"
From MIREX Wiki
| Line 3: | Line 3: | ||
==Task Descriptions== | ==Task Descriptions== | ||
| − | There will be | + | There will be '''TWO''' QBSH tasks (thus TWO '''index''' buildings and TWO '''test''' runs): |
===TASK1:=== | ===TASK1:=== | ||
| Line 9: | Line 9: | ||
Based on Prof. Jang's original idea to test for the ability to find the "ground truth" needle in a collection "haystack". | Based on Prof. Jang's original idea to test for the ability to find the "ground truth" needle in a collection "haystack". | ||
| − | * Test database: 48 ground truth MIDIs + ~2000 Essen MIDI noise files | + | * '''Test database''': 48 ground truth MIDIs + ~2000 Essen MIDI noise files |
| − | * Queries: 2719 sung queries [1] | + | * '''Queries:''' 2719 sung queries [1] |
| − | * Evaluation: Mean Reciprocal rank over top X returns | + | * '''Evaluation''': Mean Reciprocal rank over top X returns |
| Line 18: | Line 18: | ||
Based on Prof. Downie's idea that queries are variants of ground truth. | Based on Prof. Downie's idea that queries are variants of ground truth. | ||
| − | * Test database | + | * '''Test database''': 48 ground truth MIDIs + ~ 2000 Essen MIDI noise files + 2719 sung queries [1] |
| − | * Queries: 2719 sung queries + 48 ground truth MIDI files | + | * '''Queries''': 2719 sung queries + 48 ground truth MIDI files |
| − | * Evaluation: Classic precision and recall over X top returns | + | * '''Evaluation''': Classic precision and recall over X top returns |
==Moderators== | ==Moderators== | ||
| − | J.-S. Roger Jang | + | * J.-S. Roger Jang |
| − | J. Stephen Downie | + | * J. Stephen Downie |
| − | Rainer Typke | + | * Rainer Typke |
Revision as of 08:43, 21 July 2006
Overview
The QBSH: Query-by-Singing/Humming is a new task for MIREX 2006. This goal of the QBSH task is the evaluation of MIR systems that take as query input queries sung or hummed by real-world users. Prof. J.-S. Roger Jang has created a set of 2719 recorded query files representing requests for 48 different music pieces.
Task Descriptions
There will be TWO QBSH tasks (thus TWO index buildings and TWO test runs):
TASK1:
Based on Prof. Jang's original idea to test for the ability to find the "ground truth" needle in a collection "haystack".
- Test database: 48 ground truth MIDIs + ~2000 Essen MIDI noise files
- Queries: 2719 sung queries [1]
- Evaluation: Mean Reciprocal rank over top X returns
TASK2:
Based on Prof. Downie's idea that queries are variants of ground truth.
- Test database: 48 ground truth MIDIs + ~ 2000 Essen MIDI noise files + 2719 sung queries [1]
- Queries: 2719 sung queries + 48 ground truth MIDI files
- Evaluation: Classic precision and recall over X top returns
Moderators
- J.-S. Roger Jang
- J. Stephen Downie
- Rainer Typke
