Difference between revisions of "2009:Query by Tapping"
(→Indexing the MIDIs collection) |
|||
| Line 16: | Line 16: | ||
=== Jin-Da's Comments 12/08/2009 === | === Jin-Da's Comments 12/08/2009 === | ||
| − | I think we should have two subtasks for QBT. | + | I think we should have two subtasks for QBT. |
| + | <ol> | ||
| + | <li>Subtask 1: Use the onset information for comparison directly. | ||
| + | <li>Subtask 2: Use wav files for onset detection and then perform comparison. | ||
| + | </ol> | ||
== Data processing proposal for calling formats == | == Data processing proposal for calling formats == | ||
| Line 32: | Line 36: | ||
=== Running for the query files === | === Running for the query files === | ||
<pre> | <pre> | ||
| − | Running_exe <var3> <var4> <var5> | + | Usage 1: Running_exe <var3> <var4> <var5> |
| − | Running_exe <var3> <var4> | + | Usage 2: Running_exe <var3> <var4> |
where | where | ||
| Line 39: | Line 43: | ||
<var3>==<directory_of_query_rhythm_files> | <var3>==<directory_of_query_rhythm_files> | ||
<var4>==<answer_file_output_directory> | <var4>==<answer_file_output_directory> | ||
| − | <var5>==<directory_of_indexed_file> (This is | + | <var5>==<directory_of_indexed_file> (This is not required if your program does not do indexing in advance.) |
</pre> | </pre> | ||
Revision as of 00:13, 12 August 2009
Contents
Overview
The text of this section is copied from the 2008 page. Please add your comments and discussions for 2009.
The main purpose of QBT(Query by Tapping) is to evaluate MIR system in retrieval ground-truth MIDI files by the tapping rhythm. This task provides query rhythm files in WAV formate. Each wav file with a corresponding onset file for participant to evaluate by symbolic form. Evaluation database and query files can be download from http://210.68.135.13/ki/QBT.rar
Task description
- Test database: 103 ground-truth mono MIDI files.
- WAV Queries: 533 query files to retrieve 103 known target from collection. So far, 1~6 human assessors have listened and tapped a 15 seconds query rhythm from beginning for each target song. WAV files are recorded in 8k-8bit.
- Onset Queries: 533 onset symbolic query files which drawn from wav files. These onset files can help participant concentrate on similarity matching instead of onset detection. All onset files can't guarantee to have perfect detection result from original wav query files.
- Evaluation: Return top 10 candidates for each query file. 1 point is scored for a hit in the top 10 and 0 is scored otherwise (Top-10 hit rate).
Disucussions for 2009
Jin-Da's Comments 12/08/2009
We are going to have three datasets for QBT this year. Two sets are wav files (obtained from tapping on the microphone) with human labelled onset information. The other set is purely onset data obtained from computer keyboard input. Download URL of these datasets will be posted here soon.
Jin-Da's Comments 12/08/2009
I think we should have two subtasks for QBT.
- Subtask 1: Use the onset information for comparison directly.
- Subtask 2: Use wav files for onset detection and then perform comparison.
Data processing proposal for calling formats
Indexing the MIDIs collection (This is optional)
Indexing_exe <var1> <var2> where <var1>==<directory_of_MIDIs> <var2>==<indexing_files_output_and_working_directory>
Running for the query files
Usage 1: Running_exe <var3> <var4> <var5> Usage 2: Running_exe <var3> <var4> where <var3>==<directory_of_query_rhythm_files> <var4>==<answer_file_output_directory> <var5>==<directory_of_indexed_file> (This is not required if your program does not do indexing in advance.)
Data processing for output answer file formats
The answer file for each run would look like:
0001:0003,0567,0999,<insert X more responses>,XXXX 0002:0103,0567,0998,<insert X more responses>,XXXX 000X:0002,0567,0999,<insert X more responses>,XXXX
Each line represents to each of the queries in a given task run.
Submission closing date
TBA
Interested Participants
- Pierre Hanna, Matthias Robine, SIMBALS, University of Bordeaux, France. hanna at labri dot fr
- Jin-Da Chen, Jyh-Shing Roger Jang, Department of Computer Science, National Tsing-Hua University, Hsinchu, Taiwan. cdc at mirlab dot org
- Shu-Jen Show Hsiao, Tyne Liang, Department of Computer Science, National Chiao-Tung University, Hsinchu, Taiwan, R.O.C, show.cs95g at nctu.edu.tw
