Difference between revisions of "2006:Main Page"
Kahyun Choi (talk | contribs) (→Ideas for MIREX 2006 from MIREX 2005) |
|||
| (10 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
=MIREX 2006 RESULTS= | =MIREX 2006 RESULTS= | ||
| − | The results are now available. The central listing is available on the [[MIREX2006 Results]] page. The abstracts are also available [https://www.music-ir.org/ | + | The results are now available. The central listing is available on the [[2006:MIREX2006 Results]] page. The abstracts are also available [https://www.music-ir.org/mirex/abstracts/2006/MIREX2006Abstracts.pdf here]. |
| + | |||
| + | '''NOTE:''' Due to an evaluation error, the results of the [[2006:Audio Beat Tracking]] task have been updated as of 17 July 2007, and differ from those presented at ISMIR 2006. | ||
==Welcome to MIREX 2006== | ==Welcome to MIREX 2006== | ||
| Line 12: | Line 14: | ||
==MIREX 2006 Submission Instructions== | ==MIREX 2006 Submission Instructions== | ||
| − | The [[MIREX 2006 Submission Instructions]] page is now up. The URL to the MIREX 2006 Submission System has been placed at the bottom of the MIREX 2006 Submission Instructions page (to encourage reading of this important document). | + | The [[2006:MIREX 2006 Submission Instructions]] page is now up. The URL to the MIREX 2006 Submission System has been placed at the bottom of the MIREX 2006 Submission Instructions page (to encourage reading of this important document). |
| − | Andreas Ehmann has created another '''must-read''' submission-related document: [[Best Coding Practices for MIREX]]. | + | Andreas Ehmann has created another '''must-read''' submission-related document: [[2006:Best Coding Practices for MIREX]]. |
| − | Take a look at our most recent description of [[MIREX_2006_Equipment | MIREX 2006 Equipment]] and basic software. | + | Take a look at our most recent description of [[2006:MIREX_2006_Equipment | MIREX 2006 Equipment]] and basic software. |
If you have difficulties or general questions about the submission procedures please contact: | If you have difficulties or general questions about the submission procedures please contact: | ||
| Line 29: | Line 31: | ||
* '''24 July 2006''': Submission System Opened: Code and First-Pass Extended Abstract Submissions | * '''24 July 2006''': Submission System Opened: Code and First-Pass Extended Abstract Submissions | ||
* '''25 July 2006''' through '''early September''': Run evaluations (Task-specific deadlines TBA) | * '''25 July 2006''' through '''early September''': Run evaluations (Task-specific deadlines TBA) | ||
| − | * '''14 August 2006''': Deadline for [[Audio Music Similarity and Retrieval]] and [[Symbolic Melodic Similarity]] submissions. | + | * '''14 August 2006''': Deadline for [[2006:Audio Music Similarity and Retrieval]] and [[2006:Symbolic Melodic Similarity]] submissions. |
* '''25 September 2006''': Revised Abstracts Due (Tentative, depends on printing arrangements, could be earlier or later) | * '''25 September 2006''': Revised Abstracts Due (Tentative, depends on printing arrangements, could be earlier or later) | ||
===SPECIAL COMMENT ABOUT DEADLINES=== | ===SPECIAL COMMENT ABOUT DEADLINES=== | ||
| − | This year, two tasks, [[Audio Music Similarity and Retrieval]] and [[Symbolic Melodic Similarity]] will require post-run human evaluations. Therfore we are going to press hard to have the running phases of these two tasks completed by 21 August 2006. It is also important to note that both these tasks MUST HAVE each entrant run before we can move on to the human evaluation phase (i.e., no stragglers). We will be able to extend the running of the other contests later into August and early September as these involve automatic evaluation procedures and no dependencies across systems. | + | This year, two tasks, [[2006:Audio Music Similarity and Retrieval]] and [[2006:Symbolic Melodic Similarity]] will require post-run human evaluations. Therfore we are going to press hard to have the running phases of these two tasks completed by 21 August 2006. It is also important to note that both these tasks MUST HAVE each entrant run before we can move on to the human evaluation phase (i.e., no stragglers). We will be able to extend the running of the other contests later into August and early September as these involve automatic evaluation procedures and no dependencies across systems. |
| − | All runs for [[Audio Music Similarity and Retrieval]] and [[Symbolic Melodic Similarity]] must be complete by August 21st in order to take part in the human evaluation. This means that the final submission deadline falls on '''August 14th''' at the absolute latest! No submission will be considered after this date (there will be absolutely no extensions to this deadline) and your submission must actually work by the 14th (no additional debugging time). Therefore, it is highly recommended that you submit before this date and ensure that you have carefully followed all specifications. Remember this is not a flexible deadline and you should submit long before it falls. | + | All runs for [[2006:Audio Music Similarity and Retrieval]] and [[2006:Symbolic Melodic Similarity]] must be complete by August 21st in order to take part in the human evaluation. This means that the final submission deadline falls on '''August 14th''' at the absolute latest! No submission will be considered after this date (there will be absolutely no extensions to this deadline) and your submission must actually work by the 14th (no additional debugging time). Therefore, it is highly recommended that you submit before this date and ensure that you have carefully followed all specifications. Remember this is not a flexible deadline and you should submit long before it falls. |
If you have problems with the 14 August 2006 deadline please contact the MIREX team '''immediately''' to see what, if any, accommodations can be made. The more lead time we at MIREX have, the higher the probability that some accommodations might be possible. Contact: [mailto:mrx-com09@lists.lis.uiuc.edu mrx-com09@lists.lis.uiuc.edu] | If you have problems with the 14 August 2006 deadline please contact the MIREX team '''immediately''' to see what, if any, accommodations can be made. The more lead time we at MIREX have, the higher the probability that some accommodations might be possible. Contact: [mailto:mrx-com09@lists.lis.uiuc.edu mrx-com09@lists.lis.uiuc.edu] | ||
| Line 42: | Line 44: | ||
===MIREX 2006 Evaluation Tasks=== | ===MIREX 2006 Evaluation Tasks=== | ||
As of 24 July 2006, these are the tasks that we plan to run as part of MIREX 2006 | As of 24 July 2006, these are the tasks that we plan to run as part of MIREX 2006 | ||
| − | * [[Audio Beat Tracking]] | + | * [[2006:Audio Beat Tracking]] |
| − | * [[Audio Melody Extraction]] | + | * [[2006:Audio Melody Extraction]] |
| − | * [[Audio Music Similarity and Retrieval]] | + | * [[2006:Audio Music Similarity and Retrieval]] |
| − | * [[Audio Cover Song]] | + | * [[2006:Audio Cover Song]] |
| − | * [[Audio Onset Detection]] | + | * [[2006:Audio Onset Detection]] |
| − | * [[Audio Tempo Extraction]] | + | * [[2006:Audio Tempo Extraction]] |
| − | * [[QBSH: Query-by-Singing/Humming]] | + | * [[2006:QBSH: Query-by-Singing/Humming]] |
| − | * [[Score Following Proposal|Score Following]] | + | * [[2006:Score Following Proposal|Score Following]] |
| − | * [[Symbolic Melodic Similarity]] | + | * [[2006:Symbolic Melodic Similarity]] |
===Special Pairings of MIREX 2006 Tasks=== | ===Special Pairings of MIREX 2006 Tasks=== | ||
This year, we seem to have arrived at some sets of evaluation tasks that are closely related to each other in some way (i.e., similar data, similar evaluations, and/or similar algorithms that can perform both tasks, etc.). Given these pairings, do be sure to check out the related tasks to see if you might want to participate. These pairings are below. | This year, we seem to have arrived at some sets of evaluation tasks that are closely related to each other in some way (i.e., similar data, similar evaluations, and/or similar algorithms that can perform both tasks, etc.). Given these pairings, do be sure to check out the related tasks to see if you might want to participate. These pairings are below. | ||
| − | *[[Audio Beat Tracking]] and [[Audio Tempo Extraction]] | + | *[[2006:Audio Beat Tracking]] and [[2006:Audio Tempo Extraction]] |
| − | *[[Audio Music Similarity and Retrieval]] and [[Audio Cover Song]] | + | *[[2006:Audio Music Similarity and Retrieval]] and [[2006:Audio Cover Song]] |
| − | *[[Symbolic Melodic Similarity]] and [[QBSH: Query-by-Singing/Humming]] | + | *[[2006:Symbolic Melodic Similarity]] and [[2006:QBSH: Query-by-Singing/Humming]] |
==MIREX 2005 and 2006 Wikis== | ==MIREX 2005 and 2006 Wikis== | ||
| Line 66: | Line 68: | ||
Below are some ideas pilfered from the MIREX 2005 competition. As of 18 July 2006, we will NOT be running these tasks as part of MIREX 2006. We have established a 28 July 2006 DROP DEAD deadline for notifying us at IMIRSEL that at least 3 participants (representing different 3 different algorithms) are planning on participating in the running of one of these tasks. | Below are some ideas pilfered from the MIREX 2005 competition. As of 18 July 2006, we will NOT be running these tasks as part of MIREX 2006. We have established a 28 July 2006 DROP DEAD deadline for notifying us at IMIRSEL that at least 3 participants (representing different 3 different algorithms) are planning on participating in the running of one of these tasks. | ||
| − | * | + | * 2006:Audio Genre Classification |
| − | * | + | * 2006:Audio Drum Detection |
| − | * [[Melody Transcription from Monophonic Audio Recordings (it is NOT the same as melody extraction)]] | + | * [[2006:Melody Transcription from Monophonic Audio Recordings (it is NOT the same as melody extraction)]] |
| − | * [[Audio Singer Identification]] | + | * [[2006:Audio Singer Identification]] |
| − | * | + | * 2006:Audio and Symbolic Key Finding |
| − | * | + | * 2006:Symbolic Genre Classification |
===Task descriptions from [[2005:Main_Page|MIREX 2005]]=== | ===Task descriptions from [[2005:Main_Page|MIREX 2005]]=== | ||
* [[2005:Audio Artist Identification]] | * [[2005:Audio Artist Identification]] | ||
Latest revision as of 00:07, 15 December 2011
Contents
MIREX 2006 RESULTS
The results are now available. The central listing is available on the 2006:MIREX2006 Results page. The abstracts are also available here.
NOTE: Due to an evaluation error, the results of the 2006:Audio Beat Tracking task have been updated as of 17 July 2007, and differ from those presented at ISMIR 2006.
Welcome to MIREX 2006
This is the main page for the 2006 running of the Music Information Retrieval Evaluation eXchange (MIREX 2006). The International Music Information Retrieval Systems Evaluation Laboratory (IMIRSEL) at the Graduate School of Library and Information Science (GSLIS), University of Illinois at Urbana-Champaign (UIUC) is hosting and running MIREX 2006.
We wish to thank the Task Leaders who have volunteered to co-ordinate the zillion details required to conduct the scientific evaluation of the wide variety of MIR-related tasks being explored by MIR researchers from around the globe. We also wish to thank the MIR community members for their constant input, commentary and suggestions on making the evaluation tasks possible. Special thanks to the Graduate Assistants at IMIRSEL who are working ever-so-diligently to make MIREX 2006 run even better than MIREX 2005.
Cheers,
J. Stephen Downie
Director, IMIRSEL
MIREX 2006 Submission Instructions
The 2006:MIREX 2006 Submission Instructions page is now up. The URL to the MIREX 2006 Submission System has been placed at the bottom of the MIREX 2006 Submission Instructions page (to encourage reading of this important document).
Andreas Ehmann has created another must-read submission-related document: 2006:Best Coding Practices for MIREX.
Take a look at our most recent description of MIREX 2006 Equipment and basic software.
If you have difficulties or general questions about the submission procedures please contact:
MIREX 2006 at ISMIR 2006
- MIREX 2006 Poster Session: Wednesday, 11 October 2006, during the lunch/poster session from 12:00-14:00.
- MIREX Results and Planning Plenary: The plenary results and planning meeting will be held Thursday, 12 October 2006 in the afternoon after lunch so we can be more flexible with timing and the meeting can stretch longer as necessary. I suspect that we could easily go for quite some time as we really need to discuss such important issues as organizational structures and the increasingly important "lack of training/testing/ground-truth" data issue.
MIREX 2006 Preliminary Timelines
- 24 July 2006: Submission System Opened: Code and First-Pass Extended Abstract Submissions
- 25 July 2006 through early September: Run evaluations (Task-specific deadlines TBA)
- 14 August 2006: Deadline for 2006:Audio Music Similarity and Retrieval and 2006:Symbolic Melodic Similarity submissions.
- 25 September 2006: Revised Abstracts Due (Tentative, depends on printing arrangements, could be earlier or later)
SPECIAL COMMENT ABOUT DEADLINES
This year, two tasks, 2006:Audio Music Similarity and Retrieval and 2006:Symbolic Melodic Similarity will require post-run human evaluations. Therfore we are going to press hard to have the running phases of these two tasks completed by 21 August 2006. It is also important to note that both these tasks MUST HAVE each entrant run before we can move on to the human evaluation phase (i.e., no stragglers). We will be able to extend the running of the other contests later into August and early September as these involve automatic evaluation procedures and no dependencies across systems.
All runs for 2006:Audio Music Similarity and Retrieval and 2006:Symbolic Melodic Similarity must be complete by August 21st in order to take part in the human evaluation. This means that the final submission deadline falls on August 14th at the absolute latest! No submission will be considered after this date (there will be absolutely no extensions to this deadline) and your submission must actually work by the 14th (no additional debugging time). Therefore, it is highly recommended that you submit before this date and ensure that you have carefully followed all specifications. Remember this is not a flexible deadline and you should submit long before it falls.
If you have problems with the 14 August 2006 deadline please contact the MIREX team immediately to see what, if any, accommodations can be made. The more lead time we at MIREX have, the higher the probability that some accommodations might be possible. Contact: mrx-com09@lists.lis.uiuc.edu .
MIREX 2006 Evaluation Tasks
As of 24 July 2006, these are the tasks that we plan to run as part of MIREX 2006
- 2006:Audio Beat Tracking
- 2006:Audio Melody Extraction
- 2006:Audio Music Similarity and Retrieval
- 2006:Audio Cover Song
- 2006:Audio Onset Detection
- 2006:Audio Tempo Extraction
- 2006:QBSH: Query-by-Singing/Humming
- Score Following
- 2006:Symbolic Melodic Similarity
Special Pairings of MIREX 2006 Tasks
This year, we seem to have arrived at some sets of evaluation tasks that are closely related to each other in some way (i.e., similar data, similar evaluations, and/or similar algorithms that can perform both tasks, etc.). Given these pairings, do be sure to check out the related tasks to see if you might want to participate. These pairings are below.
- 2006:Audio Beat Tracking and 2006:Audio Tempo Extraction
- 2006:Audio Music Similarity and Retrieval and 2006:Audio Cover Song
- 2006:Symbolic Melodic Similarity and 2006:QBSH: Query-by-Singing/Humming
MIREX 2005 and 2006 Wikis
This is the new wiki for MIREX 2006. The wiki for MIREX 2005 is still available at https://www.music-ir.org/mirex2005/. You can interlink between the 2005 and 2006 wikis using 2005: prefix on links to link to pages in MIREX2005.
Ideas for MIREX 2006 from MIREX 2005
Below are some ideas pilfered from the MIREX 2005 competition. As of 18 July 2006, we will NOT be running these tasks as part of MIREX 2006. We have established a 28 July 2006 DROP DEAD deadline for notifying us at IMIRSEL that at least 3 participants (representing different 3 different algorithms) are planning on participating in the running of one of these tasks.
- 2006:Audio Genre Classification
- 2006:Audio Drum Detection
- 2006:Melody Transcription from Monophonic Audio Recordings (it is NOT the same as melody extraction)
- 2006:Audio Singer Identification
- 2006:Audio and Symbolic Key Finding
- 2006:Symbolic Genre Classification
Task descriptions from MIREX 2005
- 2005:Audio Artist Identification
- 2005:Audio Drum Detection
- 2005:Audio Genre Classification
- 2005:Audio Melody Extraction
- 2005:Audio Onset Detection
- 2005:Audio Tempo Extraction
- 2005:Audio and Symbolic Key Finding
- 2005:Symbolic Genre Classification
- 2005:Symbolic Melodic Similarity
--Admin 13:51, 21 July 2006 (CDT)
