Query by Singing/Humming

From MIREX Wiki
Revision as of 15:24, 4 June 2010 by Singh14 (talk | contribs)

Description

The text of this section is copied from the 2009 page. Please add your comments and discussions for 2010.

The goal of the Query-by-Singing/Humming (QBSH) task is the evaluation of MIR systems that take as query input queries sung or hummed by real-world users. More information can be found in:


Subtask 1: Classic QBSH evaluation

This is the classic QBSH problem where we need to find the ground-truth midi from a user's singing or humming.

  • Queries: human singing/humming snippets (.wav). Queries are from Roger Jang's corpus and ThinkIT corpus.
  • Database: ground-truth and noise MIDI files(which are monophonic). Comprised of 48+106 Roger Jang's and ThinkIT's ground-truth along with a cleaned version of Essen Database(2000+ MIDIs which are used last year)
  • Output: top-10 candidate list.
  • Evaluation: Top-10 hit rate (1 point is scored for a hit in the top 10 and 0 is scored otherwise).

Subtask 2: Variants QBSH evaluation

This is based on Prof. Downie's idea that queries are variants of "ground-truth" midi. In fact, this becomes more important since user-contributed singing/humming is an important part of the song database to be searched, as evidenced by the QBSH search service at www.midomi.com.

  • Queries: human singing/humming snippets (.wav). Queries are from Roger Jang's corpus and ThinkIT corpus.
  • Database: human singing/humming snippets (.wav) from all available corpora (excluding the query input being searched).
  • Output: top-10 candidate list.
  • Evaluation: Top-10 hit rate (1 point is scored for a hit in the top 10 and 0 is scored otherwise).

To make algorithms able to share intermediate steps, participants are encouraged to submit separate tracker and matcher modules instead of integrated ones, which is according to Rainer Typke's suggestion. So trackers and matchers from different submissions could work together with the same pre-defined interface and thus for us it's possible to find the best combination.


Participation in previous years

Year Participating Algorithms URL
Task1 Task2
2009 3 3 https://www.music-ir.org/mirex/wiki/2009:Query-by-Singing/Humming_Results
2008 8 9 https://www.music-ir.org/mirex/wiki/2008:Query-by-Singing/Humming_Results
2007 10 10 https://www.music-ir.org/mirex/wiki/2007:Query-by-Singing/Humming_Results
2006
Retrieved from "https://music-ir.org/mirex/w/index.php?title=Query_by_Singing/Humming&oldid=7118"