MIREX Wiki - User contributions [en]

2015:GC15UX:JDISC

2015-08-19T22:03:44Z

Chris Maden: /* Dataset */

{{DISPLAYTITLE:Grand Challenge 2015: User Experience with J-DISC}}
=Purpose=
''Holistic, user-centered evaluation of the user experience in interacting with complete, user-facing music information retrieval (MIR) systems.''

=Goals=
# ''To inspire the development of complete MIR systems.''
# ''To promote the notion of user experience as a first-class research objective in the MIR community.''

=About J-DISC=
J-DISC ([http://jdisc.columbia.edu http://jdisc.columbia.edu]) is a resource for searching and exploring jazz recordings created by the Center for Jazz Studies at Columbia University. It is organized to present complete information on jazz recording sessions, and merge a large corpus of session data into a single easily accessible repository, in a manner that can be easily searched, cross-searched, navigated and cited. In addition to the focus on recording artist/leaders of traditional discography, J-DISC incorporates extensive cultural, geographic, biographical, composer and studio information that can also be easily searched and accessed.

=Dataset=
J-DISC contains fully structured and searchable metadata. Key entities in the dataset include '''person, skill, session, track, composition, and issue'''. There are 19 tables in the dataset representing various relationships between those entities. Below are brief descriptions for each of the 19 tables (table names in alphabetical order, numbers at the end showing the number of rows in each table):

*'''composition: '''Compositions, which may be recorded as tracks at sessions. (7,104)
*'''composition_composer: '''Associations between compositions and their composers. (8,622)
* '''composition_lyricist: '''Associations between compositions and their lyricists. (880)
* ''' composition_title: '''Alternative titles for compositions. (811)
*''' issue: '''Releases or issues of tracks e.g. as albums. (545)
*'''issue_leader: '''Associations between releases and their leaders. (610)
*'''issue_overdub_people: '''Associations between overdubbed releases and people working on them. (17)
*'''issue_track: '''Associations between releases and the included tracks. (3,769)
*'''person: '''People associated with musical recordings, including musicians and composers. (5,734)
* '''person_ethnicity: '''Association of people with ethnic descriptions. (70)
*'''person_session_skill: '''Correlation between people, sessions, and skills or instruments. (21,424)
*'''person_skill: '''Correlation between people and primary skills or instruments. (6,450)
*'''person_track_skill: '''Variation from sessions in correlation between people, tracks, and skills or instruments. (7,044)
*'''session: '''Recording sessions, at which one or more musicians produced one or more tracks. (2,711)
* '''session_leader: '''Associations between sessions and their leader(s). (3,123)
* '''skill: '''Skills associated with musical recordings, including instruments played, conducting, composing. (209)
*'''track: '''Tracks laid down at recording sessions by musicians. (15,361)
*'''track_composition: '''Associations between tracks and compositions. (15,672)
*'''track_soloist: '''Associations between tracks and soloists. (630)

[[2015:GC15UX:JDISC_Schema]] presents more details about each table.

==ER Diagram of the Schema==
[[File:jdisc_schema.png|400px]]

[https://www.music-ir.org/mirex/gc15ux_jdisc/jdisc_schema.pdf Click to see the full version of the diagram (PDF)]

=Download the Dataset=
# user agreement signed during download?

=Participating Systems=
''Unlike conventional MIREX tasks, participants are not asked to submit their systems. Instead, the systems will be hosted by their developers. All participating systems need to be constructed as websites accessible to users through normal web browsers. Participating teams will submit the URLs to their systems to the GC15UX team.''

To ensure a consistent experience, evaluators will see participating systems in fixed size window: '''1024x768'''. Please test your system for this screen size.

See the [[#Evaluation Webforms]] below for a better understanding of our E6K-inpsired evaluation system design.

==Potential Participants==

Please put your names and email contacts in the following table. It is encouraged that you give your team a cool name!
{| class="wikitable"
|-
! (Cool) Team Name
! Name(s)
! Email(s)
|-
|
|
|
|-
|-
|
|
|
|-
|-
|
|
|
|-
|}

=Evaluation=
As written in the name of the Grand Challenge, the evaluation will be user-centered. All systems will be used by a number of human evaluators and be rated by them on several most important criteria in evaluating user experience.

==Criteria==

''Note that the evaluation criteria or its descriptions may be slightly changed in the months leading up to the submission deadline, as we test it and work to improve it.''

Given the GC15UX is all about how users perceive their experiences of the systems, we intend to capture the user perceptions in a minimally intrusive manner and not to burden the users/evaluators with too many questions or required data inputs. The following criteria are grounded on the literature of Human Computer Interaction (HCI) and User Experience (UX), with a careful consideration on striking a balance between being comprehensive and minimizing evaluators' cognitive load.

Evaluators will rate systems on the following criteria:

* '''Overall satisfaction''': How would you rate your overall satisfaction with the system?
Very unsatisfactory / Unsatisfactory / Slightly unsatisfactory / Neutral / Slightly satisfactory / Satisfactory / Very satisfactory

* '''Aesthetics''': How would you rate the visual attractiveness of the system?
Very Poor / Poor / Slightly Poor / Neutral / Slightly Good / Good / Excellent

* '''Ease of use''': How easy was it to figure out how to use the system?
Very difficult / Difficult / Slightly difficult / Neutral / Slightly easy / Easy / Very easy

* '''Clarity''': How well does the system communicate what is going on?
Very Poor / Poor / Slightly Poor / Neutral / Slightly Good / Good / Excellent

* '''Affordances''': How well does the system allow you to perform what you want to do?
Very Poor / Poor / Slightly Poor / Neutral / Slightly Good / Good / Excellent

* '''Performance''': Does the system work efficiently and without bugs/glitches?
Very Poor / Poor / Slightly Poor / Neutral / Slightly Good / Good / Excellent

* '''Open Text Feedback''': An open-ended question is provided for evaluators to give feedback if they wish to do so.

==Evaluators==
Evaluators will be users aged 18 and above. For this round, evaluators will be drawn primarily from the MIR community through solicitations via the ISMIR-community mailing list. The [[#Evaluation Webforms]] developed by the GC15UX team will ensure all participating systems will get equal number of evaluators.

==Tasks for Evaluators==

''To motivate the evaluators, a defined yet open task is given to the evaluators:

''You need to put together a playlist for a particular event (e.g., dinner party at your house, workout session). Try to use the assigned system to make playlists for at least a couple of different events.''

''The task is to ensure that evaluators have a (more or less) consistent goal when they interact with the systems. The goal is flexible and authentic to the evaluators' purposes ("music for their own situation"). As the task is not too specific, evaluators can potentially look for a wide range of music in terms of genre, mood and other aspects. This allows great flexibility and virtually unlimited possibility in system or service design. ''

''Another important consideration in designing the task is the music collection available for this GC15UX: the Jamando collection. Jamando music is not well-known to most users/evaluators, whereas many more commonly seen music information tasks are more or less influenced by users' familiarity to the songs and song popularity. Through this task of "finding (copyright-free) background music for a self-made video", we strive to minimize the need of looking for familiar or popular music.''

==Evaluation Results==
Statistics of the scores given by all evaluators will be reported: mean, average deviation. Meaningful text comments from the evaluators will also be reported.

==Evaluation Webforms==
Graders can take as many assignments as they wish in the My Assignments page. They are allowed to go back to the evaluation page anytime by clicking the thumbnail of the submission.

[[File:GCUX_wireframe_my_assignments.png|800px]]
 
To facilitate the evaluators and minimize their burden, the GC15UX team will provide a set of evaluation forms which wrap around the participating systems. As shown in the following image, the evaluation webforms are for scoring the participating systems, with their client interfaces embedded within an iframe in the left side of the webform.

[[File:GCUX wireframe evaluation.png|800px]]

=Organization=

==Important Dates==
This year GC15UX:JDISC adopts the two-phase model with two evaluations. The first phase will end by the ISMIR conference and we will disclose preliminary results at the conference. Then, phase II will start. Participating developers can continue improving their systems based on the feedback from the first phase and another round of evaluation will be conducted in February. We believe that this model serves the developers well since it is in accordance with the iterative nature of user-centered design. In this way, the developers will also have enough time to develop their complete MIR systems.

*July ?: announce the GC15UX:JDISC
*Sep. 28st: the first deadline for system submission
*Feb. 28st: the second deadline for system submission

==What to Submit==

A URL to the participanting system.

==Contacts==
''The GC15UX team consists of:
:J. Stephen Downie, University of Illinois (MIREX director)
:Xiao Hu, University of Hong Kong (ISMIR2014 co-chair)
:Jin Ha Lee, University of Washington (ISMIR2014 program co-chair)
:David Bainbridge, Waikato University, New Zealand
:Christopher R. Maden, University of Illinois
:Kahyun Choi, University of Illinois
:Peter Organisciak, University of Illinois
:Yun Hao, University of Illinois''

Inquiries, suggestions, questions, comments are all highly welcome! Please contact Prof. Downie [mailto:jdownie@illinois.edu] or anyone in the team.

2015:GC15UX:JDISC

2015-08-19T22:03:15Z

Chris Maden: /* About J-DISC */

{{DISPLAYTITLE:Grand Challenge 2015: User Experience with J-DISC}}
=Purpose=
''Holistic, user-centered evaluation of the user experience in interacting with complete, user-facing music information retrieval (MIR) systems.''

=Goals=
# ''To inspire the development of complete MIR systems.''
# ''To promote the notion of user experience as a first-class research objective in the MIR community.''

=About J-DISC=
J-DISC ([http://jdisc.columbia.edu http://jdisc.columbia.edu]) is a resource for searching and exploring jazz recordings created by the Center for Jazz Studies at Columbia University. It is organized to present complete information on jazz recording sessions, and merge a large corpus of session data into a single easily accessible repository, in a manner that can be easily searched, cross-searched, navigated and cited. In addition to the focus on recording artist/leaders of traditional discography, J-DISC incorporates extensive cultural, geographic, biographical, composer and studio information that can also be easily searched and accessed.

=Dataset=
J-DISC contains fully structured and searchable metadata. Key entities in the dataset include '''person, skill, session, track, composition, and issue'''. There are 19 tables in the dataset representing various relationships between those entities. Below are brief descriptions for each of the 19 tables (table names in alphabetical order, numbers at the end showing the number of rows in each table):

*'''composition: '''Compositions, which may be recorded as tracks at sessions. (7,104)
*'''composition_composer: '''Associations between compositions and their composers. (8,622)
* '''composition_lyricist: '''Associations between compositions and their lyricists. (880)
* ''' composition_title: '''Alternative titles for compositions. (811)
*''' issue: '''Releases or issues of tracks e.g. as albums. (545)
*'''issue_leader: '''Associations between releases and their leaders. (610)
*'''issue_overdub_people: '''Associations between overdubbed releases and people working on them. (17)
*'''issue_track: '''Associations between releases and the included tracks. (3,769)
*'''person: '''People associated with musical recordings, including musicians and composers. (5,734)
* '''person_ethnicity: '''Association of people with ethnic descriptions. (70)
*'''person_session_skill: '''Correlation between people, sessions, and skills or instruments. (21,424)
*'''person_skill: '''Correlation between people and primary skills or instruments. (6,450)
*'''person_track_skill: '''Variation from sessions in correlation between people, tracks, and skills or instruments. (7,044)
*'''session: '''Recording sessions, at which one or more musicians produced one or more tracks. (2,711)
* '''session_leader: '''Associations between sessions and their leader(s). (3,123)
* '''skill: '''Skills associated with musical recordings, including instruments played, conducting, composing. (209)
*'''track: '''Tracks laid down at recording sessions by musicians. (15,361)
*'''track_composition: '''Associations between tracks and compositions. (15,672)
*'''track_soloist: '''Associations between tracks and soloists. (630)

[[2015:GC15UX:JDISC_Schema]] presents more details about each table.

==ER Diagram of the Schema==
[[File:jdisc_schema.png|400px]]

[https://www.music-ir.org/mirex/gc15ux_jdisc/jdisc_schema.pdf Click to see the full version of the diagram (PDF)]

=Download the Dataset=
# user agreement signed during download?

=Participating Systems=
''Unlike conventional MIREX tasks, participants are not asked to submit their systems. Instead, the systems will be hosted by their developers. All participating systems need to be constructed as websites accessible to users through normal web browsers. Participating teams will submit the URLs to their systems to the GC15UX team.''

To ensure a consistent experience, evaluators will see participating systems in fixed size window: '''1024x768'''. Please test your system for this screen size.

See the [[#Evaluation Webforms]] below for a better understanding of our E6K-inpsired evaluation system design.

==Potential Participants==

Please put your names and email contacts in the following table. It is encouraged that you give your team a cool name!
{| class="wikitable"
|-
! (Cool) Team Name
! Name(s)
! Email(s)
|-
|
|
|
|-
|-
|
|
|
|-
|-
|
|
|
|-
|}

=Evaluation=
As written in the name of the Grand Challenge, the evaluation will be user-centered. All systems will be used by a number of human evaluators and be rated by them on several most important criteria in evaluating user experience.

==Criteria==

''Note that the evaluation criteria or its descriptions may be slightly changed in the months leading up to the submission deadline, as we test it and work to improve it.''

Given the GC15UX is all about how users perceive their experiences of the systems, we intend to capture the user perceptions in a minimally intrusive manner and not to burden the users/evaluators with too many questions or required data inputs. The following criteria are grounded on the literature of Human Computer Interaction (HCI) and User Experience (UX), with a careful consideration on striking a balance between being comprehensive and minimizing evaluators' cognitive load.

Evaluators will rate systems on the following criteria:

* '''Overall satisfaction''': How would you rate your overall satisfaction with the system?
Very unsatisfactory / Unsatisfactory / Slightly unsatisfactory / Neutral / Slightly satisfactory / Satisfactory / Very satisfactory

* '''Aesthetics''': How would you rate the visual attractiveness of the system?
Very Poor / Poor / Slightly Poor / Neutral / Slightly Good / Good / Excellent

* '''Ease of use''': How easy was it to figure out how to use the system?
Very difficult / Difficult / Slightly difficult / Neutral / Slightly easy / Easy / Very easy

* '''Clarity''': How well does the system communicate what is going on?
Very Poor / Poor / Slightly Poor / Neutral / Slightly Good / Good / Excellent

* '''Affordances''': How well does the system allow you to perform what you want to do?
Very Poor / Poor / Slightly Poor / Neutral / Slightly Good / Good / Excellent

* '''Performance''': Does the system work efficiently and without bugs/glitches?
Very Poor / Poor / Slightly Poor / Neutral / Slightly Good / Good / Excellent

* '''Open Text Feedback''': An open-ended question is provided for evaluators to give feedback if they wish to do so.

==Evaluators==
Evaluators will be users aged 18 and above. For this round, evaluators will be drawn primarily from the MIR community through solicitations via the ISMIR-community mailing list. The [[#Evaluation Webforms]] developed by the GC15UX team will ensure all participating systems will get equal number of evaluators.

==Tasks for Evaluators==

''To motivate the evaluators, a defined yet open task is given to the evaluators:

''You need to put together a playlist for a particular event (e.g., dinner party at your house, workout session). Try to use the assigned system to make playlists for at least a couple of different events.''

''The task is to ensure that evaluators have a (more or less) consistent goal when they interact with the systems. The goal is flexible and authentic to the evaluators' purposes ("music for their own situation"). As the task is not too specific, evaluators can potentially look for a wide range of music in terms of genre, mood and other aspects. This allows great flexibility and virtually unlimited possibility in system or service design. ''

''Another important consideration in designing the task is the music collection available for this GC15UX: the Jamando collection. Jamando music is not well-known to most users/evaluators, whereas many more commonly seen music information tasks are more or less influenced by users' familiarity to the songs and song popularity. Through this task of "finding (copyright-free) background music for a self-made video", we strive to minimize the need of looking for familiar or popular music.''

==Evaluation Results==
Statistics of the scores given by all evaluators will be reported: mean, average deviation. Meaningful text comments from the evaluators will also be reported.

==Evaluation Webforms==
Graders can take as many assignments as they wish in the My Assignments page. They are allowed to go back to the evaluation page anytime by clicking the thumbnail of the submission.

[[File:GCUX_wireframe_my_assignments.png|800px]]
 
To facilitate the evaluators and minimize their burden, the GC15UX team will provide a set of evaluation forms which wrap around the participating systems. As shown in the following image, the evaluation webforms are for scoring the participating systems, with their client interfaces embedded within an iframe in the left side of the webform.

[[File:GCUX wireframe evaluation.png|800px]]

=Organization=

==Important Dates==
This year GC15UX:JDISC adopts the two-phase model with two evaluations. The first phase will end by the ISMIR conference and we will disclose preliminary results at the conference. Then, phase II will start. Participating developers can continue improving their systems based on the feedback from the first phase and another round of evaluation will be conducted in February. We believe that this model serves the developers well since it is in accordance with the iterative nature of user-centered design. In this way, the developers will also have enough time to develop their complete MIR systems.

*July ?: announce the GC15UX:JDISC
*Sep. 28st: the first deadline for system submission
*Feb. 28st: the second deadline for system submission

==What to Submit==

A URL to the participanting system.

==Contacts==
''The GC15UX team consists of:
:J. Stephen Downie, University of Illinois (MIREX director)
:Xiao Hu, University of Hong Kong (ISMIR2014 co-chair)
:Jin Ha Lee, University of Washington (ISMIR2014 program co-chair)
:David Bainbridge, Waikato University, New Zealand
:Christopher R. Maden, University of Illinois
:Kahyun Choi, University of Illinois
:Peter Organisciak, University of Illinois
:Yun Hao, University of Illinois''

Inquiries, suggestions, questions, comments are all highly welcome! Please contact Prof. Downie [mailto:jdownie@illinois.edu] or anyone in the team.

MIREX HOME

2015-04-09T20:56:19Z

Chris Maden: /* MIREX 2005 - 2014 Wikis */

==Welcome to MIREX 2015==

This is the main page for the eleventh running of the Music Information Retrieval Evaluation eXchange (MIREX 2015). The International Music Information Retrieval Systems Evaluation Laboratory ([https://music-ir.org/evaluation IMIRSEL]) at the Graduate School of Library and Information Science ([http://www.lis.illinois.edu GSLIS]), University of Illinois at Urbana-Champaign ([http://www.illinois.edu UIUC]) is the principal organizer of MIREX 2015.

The MIREX 2015 community will hold its annual meeting as part of [http://ismir2015.uma.es The 16th International Conference on Music Information Retrieval], ISMIR 2015, which will be held in Malaga, Spain, October 26th-30th, 2015. The MIREX plenary and poster sessions will be held during the conference.

J. Stephen Downie 
Director, IMIRSEL 

==Task Leadership Model==

Like ISMIR 2014, we are prepared to improve the distribution of tasks for the upcoming MIREX 2015. To do so, we really need leaders to help us organize and run each task.

To volunteer to lead a task, please add your name to the "Captains" column on the new [[2015:Task Captains]] page. Please direct any communication to the [https://mail.lis.illinois.edu/mailman/listinfo/evalfest EvalFest] mailing list.

What does it mean to lead a task?
* Update wiki pages as needed
* Communicate with submitters and troubleshooting submissions
* Execution and evaluation of submissions
* Publishing final results

Due to the proprietary nature of much of the data, the submission system, evaluation framework, and most of the datasets will continue to be hosted by IMIRSEL. However, we are prepared to provide access to task organizers to manage and run submissions on the IMIRSEL systems.

We really need leaders to help us this year!

==MIREX 2015 Deadline Dates==

TBD

==MIREX 2015 Submission Instructions==
* Be sure to read through the rest of this page
* Be sure to read though the task pages for which you are submitting
* Be sure to follow the [[2009:Best Coding Practices for MIREX | Best Coding Practices for MIREX]]
* Be sure to follow the [[MIREX 2014 Submission Instructions]] including both the tutorial video and the text

==MIREX 2015 Possible Evaluation Tasks==

* [[2015:GC15UX|2015:Grand Challenge on User Experience]]
* [[2015:Audio Classification (Train/Test) Tasks]], incorporating:
** Audio US Pop Genre Classification
** Audio Latin Genre Classification
** Audio Music Mood Classification
** Audio Classical Composer Identification
** [[2015:Audio K-POP Mood Classification]]
** [[2015:Audio K-POP Genre Classification]]
* [[2015:Audio Cover Song Identification]]
* [[2015:Audio Tag Classification]]
* [[2015:Audio Music Similarity and Retrieval]]
* [[2015:Symbolic Melodic Similarity]]
* [[2015:Audio Onset Detection]]
* [[2015:Audio Key Detection]]
* [[2015:Real-time Audio to Score Alignment (a.k.a Score Following)]]
* [[2015:Query by Singing/Humming]]
* [[2015:Audio Melody Extraction]]
* [[2015:Multiple Fundamental Frequency Estimation & Tracking]]
* [[2015:Audio Chord Estimation]]
* [[2015:Query by Tapping]]
* [[2015:Audio Beat Tracking]]
* [[2015:Structural Segmentation]]
* [[2015:Audio Tempo Estimation]]
* [[2015:Discovery of Repeated Themes & Sections]]
* [[2015:Audio Downbeat Estimation]]
* [[2015:Audio Fingerprinting]]
* [[2015:Singing Voice Separation]]

===Note to New Participants===
Please take the time to read the following review articles that explain the history and structure of MIREX.

Downie, J. Stephen (2008). The Music Information Retrieval Evaluation Exchange (2005-2007): 
A window into music information retrieval research.''Acoustical Science and Technology 29'' (4): 247-255. 
Available at: [http://dx.doi.org/10.1250/ast.29.247 http://dx.doi.org/10.1250/ast.29.247]

Downie, J. Stephen, Andreas F. Ehmann, Mert Bay and M. Cameron Jones. (2010). 
The Music Information Retrieval Evaluation eXchange: Some Observations and Insights. 
''Advances in Music Information Retrieval'' Vol. 274, pp. 93-115 
Available at: [http://bit.ly/KpM5u5 http://bit.ly/KpM5u5]

===Runtime Limits===

We reserve the right to stop any process that exceeds runtime limits for each task. We will do our best to notify you in enough time to allow revisions, but this may not be possible in some cases. Please respect the published runtime limits.

===Note to All Participants===

Because MIREX is premised upon the sharing of ideas and results, '''ALL''' MIREX participants are expected to:

# submit a DRAFT 2-3 page extended abstract PDF in the ISMIR format about the submitted programme(s) to help us and the community better understand how the algorithm works when submitting their programme(s).
# submit a FINALIZED 2-3 page extended abstract PDF in the ISMIR format prior to ISMIR 2014 for posting on the respective results pages (sometimes the same abstract can be used for multiple submissions; in many cases the DRAFT and FINALIZED abstracts are the same)
# present a poster at the MIREX 2014 poster session at ISMIR 2014

===Software Dependency Requests===
If you have not submitted to MIREX before or are unsure whether IMIRSEL currently supports some of the software/architecture dependencies for your submission a [https://docs.google.com/forms/d/1ZgC72BhhN0PFFiaj9NDejRdl1eW7wAFLIyMhAtoV0CA/viewform?usp=send_form dependency request form is available]. Please submit details of your dependencies on this form and the IMIRSEL team will attempt to satisfy them for you.

Due to the high volume of submissions expected at MIREX 2015, submissions with difficult to satisfy dependencies that the team has not been given sufficient notice of may result in the submission being rejected.

Finally, you will also be expected to detail your software/architecture dependencies in a README file to be provided to the submission system.

==Getting Involved in MIREX 2015==
MIREX is a community-based endeavour. Be a part of the community and help make MIREX 2015 the best yet.

===Mailing List Participation===
If you are interested in formal MIR evaluation, you should also subscribe to the "MIREX" (aka "EvalFest") mail list and participate in the community discussions about defining and running MIREX 2015 tasks. Subscription information at:
[https://mail.lis.illinois.edu/mailman/listinfo/evalfest EvalFest Central].

If you are participating in MIREX 2015, it is VERY IMPORTANT that you are subscribed to EvalFest. Deadlines, task updates and other important information will be announced via this mailing list. Please use the EvalFest for discussion of MIREX task proposals and other MIREX related issues. This wiki (MIREX 2015 wiki) will be used to embody and disseminate task proposals, however, task related discussions should be conducted on the MIREX organization mailing list (EvalFest) rather than on this wiki, but should be summarized here.

Where possible, definitions or example code for new evaluation metrics or tasks should be provided to the IMIRSEL team who will embody them in software as part of the NEMA analytics framework, which will be released to the community at or before ISMIR 2015 - providing a standardised set of interfaces and output to disciplined evaluation procedures for a great many MIR tasks.

===Wiki Participation===
If you find that you cannot edit a MIREX wiki page, you will need to create a new account via: [[Special:Userlogin]].

Please note that because of "spam-bots", MIREX wiki registration requests may be moderated by IMIRSEL members. It might take up to 24 hours for approval (Thank you for your patience!).

==MIREX 2005 - 2014 Wikis==
Content from MIREX 2005 - 2014 are available at:
'''[[2014:Main_Page|MIREX 2014]]'''
'''[[2013:Main_Page|MIREX 2013]]'''
'''[[2012:Main_Page|MIREX 2012]]'''
'''[[2011:Main_Page|MIREX 2011]]'''
'''[[2010:Main_Page|MIREX 2010]]'''
'''[[2009:Main_Page|MIREX 2009]]'''
'''[[2008:Main_Page|MIREX 2008]]'''
'''[[2007:Main_Page|MIREX 2007]]'''
'''[[2006:Main_Page|MIREX 2006]]'''
'''[[2005:Main_Page|MIREX 2005]]'''

2014:MIREX2014 Results

2014-10-27T20:53:48Z

Chris Maden: added most of QBT results

==Results by Task ==

==OVERALL RESULTS POSTERS ==

[https://www.music-ir.org/mirex/results/2014/mirex_2014_poster.pdf MIREX 2014 Overall Results Posters (PDF)]

===Train-Test Task Set===
* [https://www.music-ir.org/nema_out/mirex2014/results/act/composer_report/ Audio Classical Composer Identification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/latin_report/ Audio Latin Genre Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/mood_report/index.html Audio Music Mood Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/mixed_report/ Audio Mixed Popular Genre Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/kgenrek_report/ Audio KPOP Genre (Annotated by Korean Annotators) Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/kgenrea_report/ Audio KPOP Genre (Annotated by American Annotators) Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/kmoodk_report/ Audio KPOP Mood (Annotated by Korean Annotators) Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/kmooda_report/ Audio KPOP Mood (Annotated by American Annotators) Classification Results ]  

===Other Tasks===

* Music Structure Segmentation Results
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/struct/mrx09/ MIREX09 dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/struct/mrx10_1/ RWC dataset - Quaero (MIREX10) Ground-truth]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/struct/mrx10_2/ RWC dataset - Original RWC Ground-truth]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/struct/sal/ SALAMI dataset]  

* [[2014:Discovery of Repeated Themes & Sections Results | Discovery of Repeated Themes & Sections Results]]

* [https://nema.lis.illinois.edu/nema_out/mirex2014/results/aod/ Audio Onset Detection Results]  

* [[2014:Audio Cover Song Identification Results|Audio Cover Song Identification Results]]

* [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ate/ Audio Tempo Estimation Results]  
* [https://nema.lis.illinois.edu/nema_out/mirex2014/results/akd/ Audio Key Detection Results]  

* Audio Beat Tracking Results
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/abt/smc/ SMC Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/abt/maz/ MAZ Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/abt/mck/ MCK Dataset]  

* Audio Chord Estimation
** [[2014:Audio_Chord_Estimation_Results_MIREX_2009 | MIREX ’09 Dataset]]  
** [[2014:Audio_Chord_Estimation_Results_Billboard_2012 | Billboard ’12 Dataset]]  
** [[2014:Audio_Chord_Estimation_Results_Billboard_2013 | Billboard ’13 Dataset]]  

* Audio Melody Extraction Results
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ame/adc04/ ADC04 Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ame/mrx05/ MIREX05 Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ame/ind08/ INDIAN08 Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ame/mrx09_0db/ MIREX09 0dB Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ame/mrx09_m5db/ MIREX09 -5dB Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ame/mrx09_p5db/ MIREX09 +5dB Dataset]  

* Audio Tag Classification Results
** Major Miner Tag dataset
*** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/atg/subtask1_report/bin/ Binary relevance (classification evaluation)]  
*** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/atg/subtask1_report/aff/ Affinity estimation evaluation]  
** Mood Tag dataset
*** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/atg/subtask2_report/bin/ Binary relevance (classification evaluation)]  
*** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/atg/subtask2_report/aff/ Affinity estimation evaluation]  

* Query-by-Singing/Humming Results
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbsh/qbsh_task1_hidden/ Hidden Jang Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbsh/qbsh_task1_jang/ Jang Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbsh/qbsh_task1_thinkit/ ThinkIt Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbsh/qbsh_task1_ioacas/ IOACAS Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbsh/qbsh_task2_jang/ Subtask2 Dataset]  

* Query-by-Tapping Results
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbt/qbt_task1_jang/ Subtask 1, Jang dataset]
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbt/qbt_task1_hsiao/ Subtask 1, Hsiao dataset]
** Subtask 1, QBT-Extended dataset
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbt/qbt_task2_jang/ Subtask 2, Jang dataset]
** Subtask 3, QBT-Extended dataset

* [[2014:Audio Downbeat Estimation Results|Audio Downbeat Estimation Results]]
* [[2014:Audio Fingerprinting Results|Audio Fingerprinting Results]]

* [https://www.music-ir.org/mirex/wiki/2014:Real-time_Audio_to_Score_Alignment_(a.k.a._Score_Following)_Results#Summary_Results Real-time Audio to Score Alignment (a.k.a. Score Following) Results]  

* [https://www.music-ir.org/mirex/wiki/2014:Audio_Music_Similarity_and_Retrieval_Results Audio Music Similarity and Retrieval Results] 
* [[2014:Symbolic_Melodic_Similarity_Results | Symbolic Melodic Similarity Results]]
* [https://www.music-ir.org/mirex/wiki/2014:Singing_Voice_Separation_Results Singing Voice Separation] 

* [https://www.music-ir.org/mirex/wiki/2014:Multiple_Fundamental_Frequency_Estimation_%26_Tracking_Results Multiple Fundamental Frequency Estimation & Tracking Results]

2014:MIREX2014 Results

2014-10-20T20:00:14Z

Chris Maden: /* Other Tasks */ added QBT result stubs

==Results by Task ==

===Train-Test Task Set===
* [https://www.music-ir.org/nema_out/mirex2014/results/act/composer_report/ Audio Classical Composer Identification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/latin_report/ Audio Latin Genre Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/mood_report/index.html Audio Music Mood Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/mixed_report/ Audio Mixed Popular Genre Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/kgenrek_report/ Audio KPOP Genre (Annotated by Korean Annotators) Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/kgenrea_report/ Audio KPOP Genre (Annotated by American Annotators) Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/kmoodk_report/ Audio KPOP Mood (Annotated by Korean Annotators) Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2014/results/act/kmooda_report/ Audio KPOP Mood (Annotated by American Annotators) Classification Results ]  

===Other Tasks===

* Music Structure Segmentation Results
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/struct/mrx09/ MIREX09 dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/struct/mrx10_1/ RWC dataset - Quaero (MIREX10) Ground-truth]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/struct/mrx10_2/ RWC dataset - Original RWC Ground-truth]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/struct/sal/ SALAMI dataset]  

* [https://nema.lis.illinois.edu/nema_out/mirex2014/results/aod/ Audio Onset Detection Results]  

* [[2014:Audio Cover Song Identification Results|Audio Cover Song Identification Results]]

* [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ate/ Audio Tempo Estimation Results]  
* [https://nema.lis.illinois.edu/nema_out/mirex2014/results/akd/ Audio Key Detection Results]  

* Audio Beat Tracking Results
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/abt/smc/ SMC Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/abt/maz/ MAZ Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/abt/mck/ MCK Dataset]  

* Audio Chord Estimation
** [[2014:Audio_Chord_Estimation_Results_MIREX_2009 | MIREX ’09 Dataset]]  
** [[2014:Audio_Chord_Estimation_Results_Billboard_2012 | Billboard ’12 Dataset]]  
** [[2014:Audio_Chord_Estimation_Results_Billboard_2013 | Billboard ’13 Dataset]]  

* Audio Melody Extraction Results
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ame/adc04/ ADC04 Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ame/mrx05/ MIREX05 Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ame/ind08/ INDIAN08 Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ame/mrx09_0db/ MIREX09 0dB Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ame/mrx09_m5db/ MIREX09 -5dB Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/ame/mrx09_p5db/ MIREX09 +5dB Dataset]  

* Audio Tag Classification Results
** Major Miner Tag dataset
*** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/atg/subtask1_report/bin/ Binary relevance (classification evaluation)]  
*** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/atg/subtask1_report/aff/ Affinity estimation evaluation]  
** Mood Tag dataset
*** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/atg/subtask2_report/bin/ Binary relevance (classification evaluation)]  
*** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/atg/subtask2_report/aff/ Affinity estimation evaluation]  

* Query-by-Singing/Humming Results
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbsh/qbsh_task1_hidden/ Hidden Jang Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbsh/qbsh_task1_jang/ Jang Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbsh/qbsh_task1_thinkit/ ThinkIt Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbsh/qbsh_task1_ioacas/ IOACAS Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2014/results/qbsh/qbsh_task2_jang/ Subtask2 Dataset]  

* Query-by-Tapping Results
** [[2014:Query by Tapping Results Subtask 1 (Jang)|Subtask 1, Jang dataset]]
** [[2014:Query by Tapping Results Subtask 1 (Hsiao)|Subtask 1, Hsiao dataset]]
** [[2014:Query by Tapping Results Subtask 2|Subtask 2, Jang dataset]]
** [[2014:Query by Tapping Results Subtask 3|Subtask 3, QBT-Extended dataset]]

* [[2014:Audio Downbeat Estimation Results|Audio Downbeat Estimation Results]]

* [https://www.music-ir.org/mirex/wiki/2014:Real-time_Audio_to_Score_Alignment_(a.k.a._Score_Following)_Results#Summary_Results Real-time Audio to Score Alignment (a.k.a. Score Following) Results]  

* [https://www.music-ir.org/mirex/wiki/2014:Audio_Music_Similarity_and_Retrieval_Results Audio Music Similarity and Retrieval Results] 
* [[2014:Symbolic_Melodic_Similarity_Results | Symbolic Melodic Similarity Results]]
* [https://www.music-ir.org/mirex/wiki/2014:Singing_Voice_Separation_Results#For_the_Music_Accompaniment_.28dB.29 Singing Voice Separation] 

* [https://www.music-ir.org/mirex/wiki/2014:Multiple_Fundamental_Frequency_Estimation_%26_Tracking_Results Multiple Fundamental Frequency Estimation & Tracking Results] 

* [[2014:Discovery of Repeated Themes & Sections Results | Discovery of Repeated Themes & Sections Results]]

2014:Query by Tapping

2014-02-13T09:44:28Z

Chris Maden: /* Test the query files */ clarifying output format

== Overview ==
The text of this section is copied from the 2013 page. Please add your comments and discussions for 2014.

The main purpose of QBT (Query by Tapping) is to evaluate MIR system in retrieving ground-truth MIDI files by tapping the onset of music notes to the microphone. This task provides query files in wave format as well as the corresponding human-label onset time in symbolic format. For this year's QBT task, we have two corpora for evaluation:

* Roger Jang's [http://mirlab.org/dataSet/public/MIR-QBT.rar MIR-QBT]: This dataset contains both wav files (recorded via microphone) and onset files (human-labeled onset time).
* Show Hsiao's [http://mirlab.org/dataSet/public/QBT_symbolic.rar QBT_symbolic]: This dataset contains only onset files (obtained from the user's tapping on keyboard).

== Task description ==

=== Subtask 1: QBT with symbolic input ===
* '''Test database''': About 150 ground-truth monophonic MIDI files in MIR-QBT/HASIO.
* '''Query files''': About 800 text files of onset time to retrieve target MIDIs in MIR_QBT/HASIO. These onset files can help participant concentrate on similarity matching instead of onset detection. All onset files cannot guarantee to have perfect detection result from original wav query files.
* '''Evaluation''': Return top 10 candidates for each query file. 1 point is scored for a hit in the top 10 and 0 is scored otherwise (Top-10 hit rate).

=== Subtask 2: QBT with wave input ===
* '''Test database''': About 150 ground-truth monophonic MIDI files in MIR-QBT.
* '''Query files''': About 800 wave files of tapping recordings to retrieve MIDIs in MIR-QBT.
* '''Evaluation''': Return top 10 candidates for each query file. 1 point is scored for a hit in the top 10 and 0 is scored otherwise (Top-10 hit rate).

== Command formats ==

=== Indexing the MIDIs collection ===
Command format should look like this:

indexing %dbMidi.list% %dir_workspace_root%

where %dbMidi.list% is the input list of database midi files named as uniq_key.mid. For example:

QBT/database/00001.mid
QBT/database/00002.mid
QBT/database/00003.mid
QBT/database/00004.mid
...

Output indexed files are placed into %dir_workspace_root%. (Note that this step is not required unless you want to index or preprocess the midi database.)

=== Test the query files ===
The command format should be like this:

qbtProgram %dbMidi_list% %query_file_list% %resultFile% %dir_workspace_root%

You can use %dir_workspace_root% to store any temporary indexing/database structures. (You can omit %dir_workspace_root% if you do not need it at all.) If the input query files are onset files (for subtask 1), then the format of %query_file_list% is like this:

qbtQuery/query_00001.onset 00001.mid
qbtQuery/query_00002.onset 00001.mid
qbtQuery/query_00003.onset 00002.mid
...

(Pleae refer to the readme.txt of the downloaded MIR-QBT corpus for the format of onset files.)

If the input query files are wave files (for subtask 2), the the format of %query_file_list% is like this:

qbtQuery/query_00001.wav 00001.mid
qbtQuery/query_00002.wav 00001.mid
qbtQuery/query_00003.wav 00002.mid
...

The result file gives top-10 candidates for each query. For instance, for wave query file, the result file should have the following format for subtask 1:

qbtQuery/query_00001.onset: 00025 01003 02200 ...
qbtQuery/query_00002.onset: 01547 02313 07653 ...
qbtQuery/query_00003.onset: 03142 00320 00973 ...
...

And for subtask 2:

qbtQuery/query_00001.wav: 00025 01003 02200 ...
qbtQuery/query_00002.wav: 01547 02313 07653 ...
qbtQuery/query_00003.wav: 03142 00320 00973 ...
...

Note that the output should be the names of the MIDI files (e.g., <code>00025</code> means <code>00025.mid</code>); they are not necessary 5-digit numbers.

== Potential Participants ==
name / email

== Discussions for 2014 ==

2013:MIREX2013 Results

2013-10-29T19:15:07Z

Chris Maden: /* Other Tasks */ reordered ABT datasets alphabetically

==OVERALL RESULTS POSTERS ==

This page is under construction.

[https://www.music-ir.org/mirex/results/2013/mirex_2013_poster.pdf MIREX 2013 Overall Results Posters (PDF)]

==Results by Task ==

===Train-Test Task Set===
* [https://www.music-ir.org/nema_out/mirex2013/results/act/composer_report/ Audio Classical Composer Identification Results ]  
* [https://www.music-ir.org/nema_out/mirex2013/results/act/latin_report/ Audio Latin Genre Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2013/results/act/mood_report/index.html Audio Music Mood Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2013/results/act/mixed_report/ Audio Mixed Popular Genre Classification Results ]  
===Other Tasks===

* Audio Beat Tracking Results
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/abt/dav/ DAV Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/abt/maz/ MAZ Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/abt/mck/ MCK Dataset]  
* Audio Chord Detection Results
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ace/mrx/ MIREX Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ace/mcg/ McGill Dataset]  
* [https://nema.lis.illinois.edu/nema_out/mirex2013/results/akd/ Audio Key Detection Results]  
* Audio Melody Extraction Results
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ame/adc04/ ADC04 Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ame/mrx05/ MIREX05 Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ame/ind08/ INDIAN08 Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ame/mrx09_0db/ MIREX09 0dB Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ame/mrx09_m5db/ MIREX09 -5dB Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ame/mrx09_p5db/ MIREX09 +5dB Dataset]  
* [[2013:Audio_Music_Similarity_and_Retrieval_Results | Audio Music Similarity and Retrieval Results]]
* [https://nema.lis.illinois.edu/nema_out/mirex2013/results/aod/ Audio Onset Detection Results]  
* Audio Tag Classification Results
** Major Miner Tag dataset
*** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/atg/subtask1_report/bin/ Binary relevance (classification evaluation)]  
*** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/atg/subtask1_report/aff/ Affinity estimation evaluation]  
** Mood Tag dataset
*** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/atg/subtask2_report/bin/ Binary relevance (classification evaluation)]  
*** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/atg/subtask2_report/aff/ Affinity estimation evaluation]  
* [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ate/ Audio Tempo Estimation Results]  
* [[2013:Multiple_Fundamental_Frequency_Estimation_&_Tracking_Results | Multiple Fundamental Frequency Estimation & Tracking Results]]
* Music Structure Segmentation Results
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/struct/mrx09/ MIREX09 dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/struct/mrx10_1/ RWC dataset - Quaero (MIREX10) Ground-truth]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/struct/mrx10_2/ RWC dataset - Original RWC Ground-truth]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/struct/sal/ SALAMI dataset]  
* Query-by-Singing/Humming Results
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbsh/qbsh_task1_hidden/ Hidden Jang Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbsh/qbsh_task1a_jang/ Jang Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbsh/qbsh_task1b_thinkit/ ThinkIt Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbsh/qbsh_task1c_ioacas/ IOACAS Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbsh/qbsh_task2_jang/ Subtask2 Dataset]  
* Query-by-Tapping Results
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbt/qbt_task1_jang/ Jang Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbt/qbt_task1_hsiao/ HSIAO Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbt/qbt_task2_jang/ Subtask2 Dataset]  
*[[2013:Real-time_Audio_to_Score_Alignment_(a.k.a._Score_Following)_Results | Real-time Audio to Score Alignment (a.k.a. Score Following) Results ]]
* [[2013:Symbolic_Melodic_Similarity_Results | Symbolic Melodic Similarity Results]]
* [[2013:Discovery of Repeated Themes & Sections Results | Discovery of Repeated Themes & Sections Results]]

2013:MIREX2013 Results

2013-10-29T19:14:31Z

Chris Maden: changed SMC to DAV under ABT

==OVERALL RESULTS POSTERS ==

This page is under construction.

[https://www.music-ir.org/mirex/results/2013/mirex_2013_poster.pdf MIREX 2013 Overall Results Posters (PDF)]

==Results by Task ==

===Train-Test Task Set===
* [https://www.music-ir.org/nema_out/mirex2013/results/act/composer_report/ Audio Classical Composer Identification Results ]  
* [https://www.music-ir.org/nema_out/mirex2013/results/act/latin_report/ Audio Latin Genre Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2013/results/act/mood_report/index.html Audio Music Mood Classification Results ]  
* [https://www.music-ir.org/nema_out/mirex2013/results/act/mixed_report/ Audio Mixed Popular Genre Classification Results ]  
===Other Tasks===

* Audio Beat Tracking Results
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/abt/dav/ DAV Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/abt/mck/ MCK Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/abt/maz/ MAZ Dataset]  
* Audio Chord Detection Results
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ace/mrx/ MIREX Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ace/mcg/ McGill Dataset]  
* [https://nema.lis.illinois.edu/nema_out/mirex2013/results/akd/ Audio Key Detection Results]  
* Audio Melody Extraction Results
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ame/adc04/ ADC04 Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ame/mrx05/ MIREX05 Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ame/ind08/ INDIAN08 Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ame/mrx09_0db/ MIREX09 0dB Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ame/mrx09_m5db/ MIREX09 -5dB Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ame/mrx09_p5db/ MIREX09 +5dB Dataset]  
* [[2013:Audio_Music_Similarity_and_Retrieval_Results | Audio Music Similarity and Retrieval Results]]
* [https://nema.lis.illinois.edu/nema_out/mirex2013/results/aod/ Audio Onset Detection Results]  
* Audio Tag Classification Results
** Major Miner Tag dataset
*** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/atg/subtask1_report/bin/ Binary relevance (classification evaluation)]  
*** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/atg/subtask1_report/aff/ Affinity estimation evaluation]  
** Mood Tag dataset
*** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/atg/subtask2_report/bin/ Binary relevance (classification evaluation)]  
*** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/atg/subtask2_report/aff/ Affinity estimation evaluation]  
* [https://nema.lis.illinois.edu/nema_out/mirex2013/results/ate/ Audio Tempo Estimation Results]  
* [[2013:Multiple_Fundamental_Frequency_Estimation_&_Tracking_Results | Multiple Fundamental Frequency Estimation & Tracking Results]]
* Music Structure Segmentation Results
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/struct/mrx09/ MIREX09 dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/struct/mrx10_1/ RWC dataset - Quaero (MIREX10) Ground-truth]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/struct/mrx10_2/ RWC dataset - Original RWC Ground-truth]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/struct/sal/ SALAMI dataset]  
* Query-by-Singing/Humming Results
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbsh/qbsh_task1_hidden/ Hidden Jang Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbsh/qbsh_task1a_jang/ Jang Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbsh/qbsh_task1b_thinkit/ ThinkIt Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbsh/qbsh_task1c_ioacas/ IOACAS Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbsh/qbsh_task2_jang/ Subtask2 Dataset]  
* Query-by-Tapping Results
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbt/qbt_task1_jang/ Jang Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbt/qbt_task1_hsiao/ HSIAO Dataset]  
** [https://nema.lis.illinois.edu/nema_out/mirex2013/results/qbt/qbt_task2_jang/ Subtask2 Dataset]  
*[[2013:Real-time_Audio_to_Score_Alignment_(a.k.a._Score_Following)_Results | Real-time Audio to Score Alignment (a.k.a. Score Following) Results ]]
* [[2013:Symbolic_Melodic_Similarity_Results | Symbolic Melodic Similarity Results]]
* [[2013:Discovery of Repeated Themes & Sections Results | Discovery of Repeated Themes & Sections Results]]