MIREX Wiki - User contributions [en]

User:Tia

2026-07-12T20:07:11Z

Junyan: Creating user page for new user.

Task captain for MIREX 2026:Music Audio Generation

User talk:Tia

2026-07-12T20:07:11Z

Junyan: Welcome!

'''Welcome to ''MIREX Wiki''!'''
We hope you will contribute much and well.
You will probably want to read the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents help pages].
Again, welcome and have fun! [[User:Junyan|Junyan]] ([[User talk:Junyan|talk]]) 15:07, 12 July 2026 (CDT)

MIREX HOME

2026-07-03T12:28:57Z

Junyan: /* Task Descriptions */

==Welcome to MIREX 2026==

MIREX (Music Information Retrieval Evaluation eXchange) is an annual community evaluation campaign held in conjunction with the [https://ismir.net/ International Society for Music Information Retrieval Conference (ISMIR)]. This year, the conference will be held in [https://ismir2026.ismir.net/ Abu Dhabi, UAE] from November 8–12, 2026, and may include an online component.

In a long run, we want to make MIREX a platform for researchers to share their latest research results, to compare their systems with others, and to promote the development of the field.

==Task Descriptions==

Traditional MIR tasks
* [[2026:Lyrics Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Audio Instrument Recognition]] <TC: [mailto:wenye.ma@mail.mcgill.ca Wenye Ma]>
* Standardized Evaluation Suites <TC: [mailto:jj2731@nyu.edu Junyan Jiang] & [mailto:akira.maezawa@music.yamaha.com Akira Maezawa]>
** [[2026:Audio Chord Estimation]]
** [[2026:Audio Beat Tracking]]
** [[2026:Audio Key Detection]]
** [[2026:Audio Downbeat Estimation]]
** [[2026:Music Structure Analysis]]
** [[2026:Lyrics-to-Audio Alignment]]

Modern MIR Tasks
* [[2026:Symbolic Music Generation]] <TC: [mailto:ziyu.wang@nyu.edu Ziyu Wang], [mailto:Xinyue.Li@mbzuai.ac.ae Xinyue Li] [mailto:jzhao@u.nus.edu Jingwei Zhao]>
* [[2026:Audio-to-Score Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Rhythm Game Chart Generation]] <TC: [mailto:ziyun.liu@univ-lille.fr Ziyun Liu] & [mailto:carolina.carusi@kaist.ac.kr Carolina Carusi]>
* [[2026:Music Evaluation via CMI-RewardBench]] <TC: [mailto:yinghao.ma@qmul.ac.uk Yinghao Ma], Yonghyun Kim, Haiwen Xia, Junwon Lee>
* [[2026:AI-Generated Music Detection]] <TC: [mailto:ldzhangyx@outlook.com Yixiao Zhang] & [mailto:you.zhang@rochester.edu Neil Zhang]>
* [[2026:Music Performance Difficulty Prediction]] <TC: [mailto:huan.zhang@qmul.ac.uk Huan Zhang] & [mailto:pedro@songscription.ai Pedro Ramoneda]>

Possible Tasks
* 2026:Music Audio Generation <TC: TBD>

==Call for Challenges==

We invite the ISMIR community to propose new research challenges that address cutting-edge problems in Music Information Retrieval (MIR). These challenges should aim to push the boundaries of current research and foster innovation in the field. We also welcome challenge sponsors from both industry and research institutions, particularly those willing to contribute datasets and computational resources to support the competition.

For the format and requirements for the challenge proposal, please go to [[2026:Call for Challenges]].

Task Captain Responsibilities:

* Register on the [https://www.music-ir.org/mirex MIREX Wiki] and maintain a task description page.
* Collect submissions via the MIREX submission server (or provide customized submission instructions).
* Execute and evaluate the submissions.
* Report results to MIREX and create a results page on the MIREX Wiki.
* (Optional) Present a MIREX task captain poster at the Late-Breaking and Demo (LBD) session at ISMIR 2026.

===What's new:===

In MIREX 2026, we introduce two special calls for challenges:

1. Novel evaluation pipelines

There is a growing need for new evaluation methodologies. We welcome challenge proposals that introduce novel evaluation pipelines, including but not limited to the evaluation of generative models, interactive systems, and other emerging paradigms.

2. Evaluation under limited resources

In addition to state-of-the-art systems, MIREX aims to better support researchers and research areas with limited resources. We welcome challenge proposals that focus on underrepresented research areas, music genres, or research communities.

Furthermore, all task captains are encouraged to report key resource indicators for each submission, including:

* training data size
* model size
* other computational resources used in model development

==How to Participate==

See also the general [[Submission Guidelines]].

* Read the [[Participant Agreement]] and task description carefully.
* Program your system.
* Write a 2-4 page extended abstract PDF describing your system.
* Submit your system and extended abstract to the [http://futuremirex.com/submission MIREX submission site].
* Top-performing teams will have the opportunity to present their MIREX posters at the LBD session at ISMIR 2026.

==Important Dates==

* Challenge proposals due: May 15, 2026
* Notification of acceptance: May 29, 2026
* Submission open: Jul 1, 2026
* Submission close: Oct 1, 2026 (Some tasks may have a different deadline; see task descriptions)
* Result published: Oct 15, 2026 (Some tasks may have a different deadline; see task descriptions)

==Contact Us==

====Email====

For general questions, feedback, and suggestions, please send messages to our mailing list [mailto:future-mirex@googlegroups.com future-mirex@googlegroups.com].

For task-specific questions, we have listed the email for each task captain [[MIREX_HOME#Task_Descriptions|here]].

====Discord Server====

For real-time discussion with the MIREX organizers or task captains, you may join our [https://discord.gg/vC2YWX29sC discord server].

Notice: some task captains are not in the discord server.

====Repositories====

Open-source evaluation pipelines: https://github.com/ismir-mirex/mirex-evaluation

Github organization: https://github.com/ismir-mirex/

====LinkedIn Organization Page====

You may visit our LinkedIn organization page [https://www.linkedin.com/company/future-mirex/ here].

We are looking forward to seeing you at MIREX 2026!

Future MIREX Team, 2026

MIREX 2026 Organizers:
* Junyan Jiang, New York University
* Gus Xia, MBZUAI
* Akira Maezawa, Yamaha
* Ziyu Wang, New York University
* Yixiao Zhang, ByteDance Inc.
* Ruibin Yuan, Hong Kong University of Science and Technology
* J. Stephen Downie, University of Illinois

2026:Music Structure Analysis

2026-07-02T16:49:15Z

Junyan: /* Baseline */

== Music Structure Analysis (MIREX 2026) ==

Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation].

=== Description ===

The aim of the MIREX Music Structure Analysis task is to identify and label key structural sections in musical audio. Understanding the musical form (e.g., intro, verse, chorus) is fundamental to music understanding and a crucial component in many music information retrieval applications. While traditional approaches focused on segmenting music into internally consistent, but arbitrarily labeled, sections (e.g., A, B, C), this task has evolved.

Since 2020, a new paradigm has emerged, focusing on '''functional structure analysis'''. The goal is to segment the audio and assign a specific functional label to each segment from a predefined set of common musical functions. This task challenges systems to perform both accurate boundary detection and correct functional classification.

This task builds upon a history of structural segmentation evaluations, first run in MIREX 2009. Recent works driving this updated focus include:
* Wang, J. C., Hung, Y. N., & Smith, J. B. (2022, May). To catch a chorus, verse, intro, or anything else: Analyzing a song with structural functions. In ''ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)'' (pp. 416-420). IEEE.
* Kim, T., & Nam, J. (2023, October). All-in-one metrical and functional structure analysis with neighborhood attentions on demixed audio. In ''2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)'' (pp. 1-5). IEEE.
* Buisson, M., McFee, B., Essid, S., & Crayencour, H. C. (2024). Self-supervised learning of multi-level audio representations for music segmentation. ''IEEE/ACM Transactions on Audio, Speech, and Language Processing''.

For MIREX 2026, participants are required to segment musical audio and classify each segment into one of seven functional categories: '''‘intro’, ‘verse’, ‘chorus’, ‘bridge’, ‘inst’ (instrumental), ‘outro’, or ‘other’'''. The 'other' category can be used for segments that do not fit into the primary six functional labels or for non-musical content if explicitly defined by the dataset annotations being mapped.

=== Data ===

Other collections may be used, but participants must clearly state their dataset usage and, when applicable, the data collection process in the extended abstract. The Harmonix Set train and validation splits may also be used for training and development, but the Harmonix Set test split is held out.

We use the relabeled Harmonix Set for evaluation. The test split is held out for evaluation, and participants may use the train and validation splits for training models.

https://huggingface.co/datasets/m-a-p/harmonixset_bigvgan/tree/main

==== Collections ====
The evaluation will utilize datasets previously established in MIREX. Annotations from these diverse collections will be mapped to the seven target functional labels for consistent evaluation.
* '''The MIREX 2009 Collection''': 297 pieces, largely derived from the work of the Beatles.
* '''MIREX 2010 RWC collection''': 100 pieces of popular music. This collection has two sets of ground truths. The first was originally included with the RWC dataset. The second set provides segment boundary annotations (see [http://hal.inria.fr/docs/00/47/34/79/PDF/PI-1948.pdf Pechuho et al., 2010] for details).
* '''MIREX 2012 dataset''': Over 1,000 annotated pieces covering a range of musical styles, with the majority annotated by two independent annotators.

Participants should be aware that original labels in these datasets (e.g., 'verse1', 'solo', 'fade-out') will need to be mapped to the seven specified functional categories for evaluation. Guidelines for this mapping will be provided, or a standard mapping will be applied during evaluation.

==== Audio Formats (Input to Algorithms) ====
Algorithms should be prepared to process audio with the following characteristics:
* Sample rate: 44.1 kHz
* Bit depth: 16 bit
* Number of channels: 1 (mono)
* Encoding: WAV

=== Submission Format ===

Submissions will be handled via '''CodeBench'''. Participants are required to submit their results in a specific format, as detailed below. You will upload a single file containing the segmentation results for all test audio files.

==== Output Data Format ====
The output must be a '''list of dictionaries''' in a text-based format (e.g., JSON parsable). Each dictionary in the list corresponds to one audio file and must contain two keys: <tt>'id'</tt> (the identifier of the audio file, e.g., '1.wav') and <tt>'result'</tt> (a list of segment predictions). Each segment prediction is a list containing two elements: a two-element list with the <tt>[start_time, end_time]</tt> of the segment in seconds, and the <tt>label</tt> string for that segment.

The labels must be one of the seven target functional categories: <tt>'intro'</tt>, <tt>'verse'</tt>, <tt>'chorus'</tt>, <tt>'bridge'</tt>, <tt>'inst'</tt>, <tt>'outro'</tt>, <tt>'silence'</tt>.

Example of the content of the submitted file:
<pre>
[
{
'id': 'track01.wav',
'result': [
[[0.000, 15.500], 'intro'],
[[15.500, 45.230], 'verse'],
[[45.230, 75.800], 'chorus'],
[[75.800, 90.000], 'outro']
]
},
{
'id': 'track02.wav',
'result': [
[[0.000, 20.100], 'verse'],
[[20.100, 38.500], 'chorus'],
[[38.500, 55.000], 'verse'],
[[55.000, 72.600], 'chorus'],
[[72.600, 89.000], 'bridge'],
[[89.000, 105.000], 'chorus'],
[[105.000, 115.500], 'outro']
]
}
]
</pre>
Ensure that <tt>offset_time</tt> of one segment is the <tt>onset_time</tt> of the next segment, and segments cover the entire duration of the piece analyzed. The first segment must start at <tt>0.0</tt>.

=== Evaluation Procedures ===

Evaluation will focus on both the accuracy of the detected segment boundaries and the correctness of the assigned functional labels. The primary metrics are:

# '''Frame-Level Accuracy (ACC)''':
# Both the system output and the ground truth will be converted into time-series of labels at a fine temporal resolution (e.g., 10ms or 100ms frames). Accuracy is calculated as the proportion of frames that are correctly labeled by the system compared to the ground truth across the entire dataset. This metric evaluates the overall correctness of segment labels and their temporal extents.

# '''Boundary Retrieval Hit Rate F-Measures (HR.5F and HR3F)''':
# This metric assesses the system's ability to correctly identify segment boundaries.
# * A predicted boundary is considered a '''hit''' if it falls within a certain tolerance window of a ground truth boundary.
# * Two tolerance windows will be used:
# ** 0.5 seconds: For finer precision.
# ** 3.0 seconds: For coarser, more perceptually relevant boundaries.
# * Based on these hits, '''Precision (P)''', '''Recall (R)''', and '''F-measure (F1-score)''' will be calculated for boundary detection at both tolerance levels.
# <math>P = \frac{\text{Number of correctly retrieved boundaries}}{\text{Total number of retrieved boundaries}}</math>
# <math>R = \frac{\text{Number of correctly retrieved boundaries}}{\text{Total number of ground truth boundaries}}</math>
# <math>F = \frac{2 \times P \times R}{P + R}</math>
# * The reported metrics will be '''HR.5F''' (F-measure with 0.5s tolerance) and '''HR3F''' (F-measure with 3s tolerance).

==== Baseline ====
The performance of the method described in '''Kim, T., & Nam, J. (2023). All-in-one metrical and functional structure analysis with neighborhood attentions on demixed audio.''' will serve as a baseline for this task. Participants are encouraged to develop systems that surpass this baseline.

A new baseline, SongFormer, will also be used this year: https://arxiv.org/pdf/2510.02797

=== Relevant Development Collections ===
While the MIREX datasets will be used for evaluation, participants may find the following publicly available annotated corpora useful for development. Please note that the annotations in these corpora will also need to be mapped to the 7-class functional labeling scheme if used for training models for this task.

* Jouni Paulus's [http://www.cs.tut.fi/sgn/arg/paulus/structure.html structure analysis page] links to a corpus of 177 Beatles songs ([http://www.cs.tut.fi/sgn/arg/paulus/beatles_sections_TUT.zip zip file]). The TUTstructure07 dataset, containing 557 songs, is also listed [http://www.cs.tut.fi/sgn/arg/paulus/TUTstructure07_files.html here].
* Ewald Peiszer's [http://www.ifs.tuwien.ac.at/mir/audiosegmentation.html thesis page] links to a portion of his corpus: 43 non-Beatles pop songs (including 10 J-pop songs) ([http://www.ifs.tuwien.ac.at/mir/audiosegmentation/dl/ep_groundtruth_excl_Paulus.zip zip file]).

These public corpora offer over 200 songs that can be adapted for development purposes.

=== Time and Hardware Limits ===
Due to the nature of the CodeBench platform and the potentially high number of participants, limits on the runtime and computational resources for submissions may be imposed. Specific details regarding these limits will be provided closer to the submission deadline. A general guideline is that analysis should be computationally feasible. For reference, a hard limit of '''24 hours''' for total analysis time over the evaluation dataset was imposed in previous iterations, and a similar constraint might apply.

MIREX HOME

2026-07-01T00:54:10Z

Junyan: /* Task Descriptions */

==Welcome to MIREX 2026==

MIREX (Music Information Retrieval Evaluation eXchange) is an annual community evaluation campaign held in conjunction with the [https://ismir.net/ International Society for Music Information Retrieval Conference (ISMIR)]. This year, the conference will be held in [https://ismir2026.ismir.net/ Abu Dhabi, UAE] from November 8–12, 2026, and may include an online component.

In a long run, we want to make MIREX a platform for researchers to share their latest research results, to compare their systems with others, and to promote the development of the field.

==Task Descriptions==

Traditional MIR tasks
* [[2026:Lyrics Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Audio Instrument Recognition]] <TC: [mailto:wenye.ma@mail.mcgill.ca Wenye Ma]>
* Standardized Evaluation Suites <TC: [mailto:jj2731@nyu.edu Junyan Jiang] & [mailto:akira.maezawa@music.yamaha.com Akira Maezawa]>
** [[2026:Audio Chord Estimation]]
** [[2026:Audio Beat Tracking]]
** [[2026:Audio Key Detection]]
** [[2026:Audio Downbeat Estimation]]
** [[2026:Music Structure Analysis]]
** [[2026:Lyrics-to-Audio Alignment]]

Modern MIR Tasks
* [[2026:Symbolic Music Generation]] <TC: [mailto:ziyu.wang@nyu.edu Ziyu Wang], [mailto:Xinyue.Li@mbzuai.ac.ae Xinyue Li] [mailto:jzhao@u.nus.edu Jingwei Zhao]>
* [[2026:Audio-to-Score Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Rhythm Game Chart Generation]] <TC: [mailto:ziyun.liu@univ-lille.fr Ziyun Liu] & [mailto:carolina.carusi@kaist.ac.kr Carolina Carusi]>
* [[2026:Music Evaluation via CMI-RewardBench]] <TC: [mailto:yinghao.ma@qmul.ac.uk Yinghao Ma], Yonghyun Kim, Haiwen Xia, Junwon Lee>
* [[2026:AI-Generated Music Detection]] <TC: [mailto:ldzhangyx@outlook.com Yixiao Zhang] & [mailto:you.zhang@rochester.edu Neil Zhang]>
* [[2026:Music Performance Difficulty Prediction]] <TC: [mailto:huan.zhang@qmul.ac.uk Huan Zhang] & [mailto:pedro@songscription.ai Pedro Ramoneda]>

==Call for Challenges==

We invite the ISMIR community to propose new research challenges that address cutting-edge problems in Music Information Retrieval (MIR). These challenges should aim to push the boundaries of current research and foster innovation in the field. We also welcome challenge sponsors from both industry and research institutions, particularly those willing to contribute datasets and computational resources to support the competition.

For the format and requirements for the challenge proposal, please go to [[2026:Call for Challenges]].

Task Captain Responsibilities:

* Register on the [https://www.music-ir.org/mirex MIREX Wiki] and maintain a task description page.
* Collect submissions via the MIREX submission server (or provide customized submission instructions).
* Execute and evaluate the submissions.
* Report results to MIREX and create a results page on the MIREX Wiki.
* (Optional) Present a MIREX task captain poster at the Late-Breaking and Demo (LBD) session at ISMIR 2026.

===What's new:===

In MIREX 2026, we introduce two special calls for challenges:

1. Novel evaluation pipelines

There is a growing need for new evaluation methodologies. We welcome challenge proposals that introduce novel evaluation pipelines, including but not limited to the evaluation of generative models, interactive systems, and other emerging paradigms.

2. Evaluation under limited resources

In addition to state-of-the-art systems, MIREX aims to better support researchers and research areas with limited resources. We welcome challenge proposals that focus on underrepresented research areas, music genres, or research communities.

Furthermore, all task captains are encouraged to report key resource indicators for each submission, including:

* training data size
* model size
* other computational resources used in model development

==How to Participate==

See also the general [[Submission Guidelines]].

* Read the [[Participant Agreement]] and task description carefully.
* Program your system.
* Write a 2-4 page extended abstract PDF describing your system.
* Submit your system and extended abstract to the [http://futuremirex.com/submission MIREX submission site].
* Top-performing teams will have the opportunity to present their MIREX posters at the LBD session at ISMIR 2026.

==Important Dates==

* Challenge proposals due: May 15, 2026
* Notification of acceptance: May 29, 2026
* Submission open: Jul 1, 2026
* Submission close: Oct 1, 2026 (Some tasks may have a different deadline; see task descriptions)
* Result published: Oct 15, 2026 (Some tasks may have a different deadline; see task descriptions)

==Contact Us==

====Email====

For general questions, feedback, and suggestions, please send messages to our mailing list [mailto:future-mirex@googlegroups.com future-mirex@googlegroups.com].

For task-specific questions, we have listed the email for each task captain [[MIREX_HOME#Task_Descriptions|here]].

====Discord Server====

For real-time discussion with the MIREX organizers or task captains, you may join our [https://discord.gg/vC2YWX29sC discord server].

Notice: some task captains are not in the discord server.

====Repositories====

Open-source evaluation pipelines: https://github.com/ismir-mirex/mirex-evaluation

Github organization: https://github.com/ismir-mirex/

====LinkedIn Organization Page====

You may visit our LinkedIn organization page [https://www.linkedin.com/company/future-mirex/ here].

We are looking forward to seeing you at MIREX 2026!

Future MIREX Team, 2026

MIREX 2026 Organizers:
* Junyan Jiang, New York University
* Gus Xia, MBZUAI
* Akira Maezawa, Yamaha
* Ziyu Wang, New York University
* Yixiao Zhang, ByteDance Inc.
* Ruibin Yuan, Hong Kong University of Science and Technology
* J. Stephen Downie, University of Illinois

MIREX HOME

2026-07-01T00:52:29Z

Junyan:

==Welcome to MIREX 2026==

MIREX (Music Information Retrieval Evaluation eXchange) is an annual community evaluation campaign held in conjunction with the [https://ismir.net/ International Society for Music Information Retrieval Conference (ISMIR)]. This year, the conference will be held in [https://ismir2026.ismir.net/ Abu Dhabi, UAE] from November 8–12, 2026, and may include an online component.

In a long run, we want to make MIREX a platform for researchers to share their latest research results, to compare their systems with others, and to promote the development of the field.

==Task Descriptions==

Traditional MIR tasks
* [[2026:Lyrics Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Audio Instrument Recognition]] <TC: [mailto:wenye.ma@mail.mcgill.ca Wenye Ma]>
* Standardized Evaluation Suites <TC: [mailto:jj2731@nyu.edu Junyan Jiang] & Akira Maezawa>
* [[2026:Audio Chord Estimation]]
* [[2026:Audio Beat Tracking]]
* [[2026:Audio Key Detection]]
* [[2026:Audio Downbeat Estimation]]
* [[2026:Music Structure Analysis]]
* [[2026:Lyrics-to-Audio Alignment]]

Modern MIR Tasks
* [[2026:Symbolic Music Generation]] <TC: [mailto:ziyu.wang@nyu.edu Ziyu Wang], [mailto:Xinyue.Li@mbzuai.ac.ae Xinyue Li] [mailto:jzhao@u.nus.edu Jingwei Zhao]>
* [[2026:Audio-to-Score Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Rhythm Game Chart Generation]] <TC: [mailto:ziyun.liu@univ-lille.fr Ziyun Liu] & [mailto:carolina.carusi@kaist.ac.kr Carolina Carusi]>
* [[2026:Music Evaluation via CMI-RewardBench]] <TC: [mailto:yinghao.ma@qmul.ac.uk Yinghao Ma], Yonghyun Kim, Haiwen Xia, Junwon Lee>
* [[2026:AI-Generated Music Detection]] <TC: [mailto:ldzhangyx@outlook.com Yixiao Zhang] & [mailto:you.zhang@rochester.edu Neil Zhang]>
* [[2026:Music Performance Difficulty Prediction]] <TC: [mailto:huan.zhang@qmul.ac.uk Huan Zhang] & [mailto:pedro@songscription.ai Pedro Ramoneda]>

==Call for Challenges==

We invite the ISMIR community to propose new research challenges that address cutting-edge problems in Music Information Retrieval (MIR). These challenges should aim to push the boundaries of current research and foster innovation in the field. We also welcome challenge sponsors from both industry and research institutions, particularly those willing to contribute datasets and computational resources to support the competition.

For the format and requirements for the challenge proposal, please go to [[2026:Call for Challenges]].

Task Captain Responsibilities:

* Register on the [https://www.music-ir.org/mirex MIREX Wiki] and maintain a task description page.
* Collect submissions via the MIREX submission server (or provide customized submission instructions).
* Execute and evaluate the submissions.
* Report results to MIREX and create a results page on the MIREX Wiki.
* (Optional) Present a MIREX task captain poster at the Late-Breaking and Demo (LBD) session at ISMIR 2026.

===What's new:===

In MIREX 2026, we introduce two special calls for challenges:

1. Novel evaluation pipelines

There is a growing need for new evaluation methodologies. We welcome challenge proposals that introduce novel evaluation pipelines, including but not limited to the evaluation of generative models, interactive systems, and other emerging paradigms.

2. Evaluation under limited resources

In addition to state-of-the-art systems, MIREX aims to better support researchers and research areas with limited resources. We welcome challenge proposals that focus on underrepresented research areas, music genres, or research communities.

Furthermore, all task captains are encouraged to report key resource indicators for each submission, including:

* training data size
* model size
* other computational resources used in model development

==How to Participate==

See also the general [[Submission Guidelines]].

* Read the [[Participant Agreement]] and task description carefully.
* Program your system.
* Write a 2-4 page extended abstract PDF describing your system.
* Submit your system and extended abstract to the [http://futuremirex.com/submission MIREX submission site].
* Top-performing teams will have the opportunity to present their MIREX posters at the LBD session at ISMIR 2026.

==Important Dates==

* Challenge proposals due: May 15, 2026
* Notification of acceptance: May 29, 2026
* Submission open: Jul 1, 2026
* Submission close: Oct 1, 2026 (Some tasks may have a different deadline; see task descriptions)
* Result published: Oct 15, 2026 (Some tasks may have a different deadline; see task descriptions)

==Contact Us==

====Email====

For general questions, feedback, and suggestions, please send messages to our mailing list [mailto:future-mirex@googlegroups.com future-mirex@googlegroups.com].

For task-specific questions, we have listed the email for each task captain [[MIREX_HOME#Task_Descriptions|here]].

====Discord Server====

For real-time discussion with the MIREX organizers or task captains, you may join our [https://discord.gg/vC2YWX29sC discord server].

Notice: some task captains are not in the discord server.

====Repositories====

Open-source evaluation pipelines: https://github.com/ismir-mirex/mirex-evaluation

Github organization: https://github.com/ismir-mirex/

====LinkedIn Organization Page====

You may visit our LinkedIn organization page [https://www.linkedin.com/company/future-mirex/ here].

We are looking forward to seeing you at MIREX 2026!

Future MIREX Team, 2026

MIREX 2026 Organizers:
* Junyan Jiang, New York University
* Gus Xia, MBZUAI
* Akira Maezawa, Yamaha
* Ziyu Wang, New York University
* Yixiao Zhang, ByteDance Inc.
* Ruibin Yuan, Hong Kong University of Science and Technology
* J. Stephen Downie, University of Illinois

2026:Lyrics-to-Audio Alignment

2026-06-29T19:31:13Z

Junyan: Created page with "==Description== Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation]. The task of automatic lyrics-to-audio alignm..."

==Description==

Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation].

The task of automatic lyrics-to-audio alignment has as an end goal the synchronization between an audio recording of singing and its corresponding written lyrics. The beginning timestamps of lyrics units can be estimated on different granularity: phonemes, words, lyrics lines, phrases. For this task word-level alignment is required.

----------------------- ---------------------------------------------------
| Mixed singing audio | | Lyrics at word-level: no more carefree ... ... |
----------------------- ---------------------------------------------------
| |
--------------------------------------------
|
--------------------
| Alignment system |
--------------------
|
|
--------------------------
| 0.123 0.798 no |
| 0.798 1.123 more |
| 1.345 2.176 carefree|
| ... ... |
--------------------------
The algorithm receives two inputs - mixed singing audio (singing voice + musical accompaniment) and its corresponding lyrics at word-level, outputs the onset and offset timestamps (second) of each word.

== What's New ==

Compared to previous years:

* Submission format: docker image is required. See the submission format section.

== Data ==

=== Test Data ===

Participants are not allowed to use any split of the following datasets for training, validation, model selection, parameter tuning, or any other development purpose:

'''Jamendo''': This dataset contains 20 full-duration music pieces with 10 different Western music genres, annotated with start-of-word timestamps. All songs have instrumental accompaniment. It is available online on [https://github.com/f90/jamendolyrics Github]. Because Jamendo is held out for this task, it must not be used for training, validation, model selection, or parameter tuning. For more information also refer to [https://arxiv.org/abs/1902.06797 this paper].

* file duration up to 4:43 (total time: 1h 12m)
* 5677 words annotated in total

=== Other Collections ===

Other collections may be used, but participants must clearly state their dataset usage and, when applicable, the data collection process in the extended abstract.

==== DAMP dataset ====
The DAMP dataset contains a large number (34 000+) of a cappella recordings from a wide variety of amateur singers, collected with the Sing! Karaoke mobile app in different recording conditions, but generally with good audio quality. A carefully curated subset DAMPB of 20 performances of each of the 300 songs has been created by (Kruspe, 2016). Here is the [https://docs.google.com/spreadsheets/d/1YwhPhXU6t-BMZfdEODS_pNW_umFIsciYL62kh-fiBWI/edit?usp=sharing list of recordings].

* The audio can be downloaded from the [https://ccrma.stanford.edu/damp/ Smule web site]
* No lyrics boundary annotations are available, still the textual lyrics are on the [https://www.smule.com/songs Smule Sing! Karaoke website]

==== DALI Dataset ====

The DALI dataset (a large '''D'''ataset of synchronised '''A'''udio, '''L'''yr'''I'''cs and notes) contains over 5000 songs with semi-automatically aligned lyrics annotations. The songs are commercial recordings in full-duration, whereas the lyrics are described according to different levels of granularity including words and notes (and syllables underlying a given note). For each song DALI provides a link to a matched youtube video, from which the audio could be retrieved.
For more details how, see its full description [https://github.com/gabolsgabs/DALI here].

==== Hansen's Dataset ====
The dataset contains 9 pop music songs in English with annotations of both beginnings- and ending-timestamps of each word. The ending timestamps are for convenience (copies of next word's beginning timestamp) and are not used in the evaluation. Sentence-level annotations are also provided.
The audio has two versions: the original mix with instrumental accompaniment and a cappella singing voice only one. An example song can be seen [https://www.dropbox.com/sh/wm6k4dqrww0fket/AAC1o1uRFxBPg9iAeSAd1Wxta?dl=0 here]

You can read in detail about how the dataset was made here: [http://smcnetwork.org/system/files/smc2012-198.pdf Recognition of Phonemes in A-cappella Recordings using Temporal Patterns and Mel Frequency Cepstral Coefficients]. The dataset has been kindly provided by Jens Kofod Hansen.

* file duration up to 4:40 minutes (total time: 35:33 minutes)
* 3590 words annotated in total

==== Mauch's Dataset ====
The dataset contains 20 pop music songs in English with annotations of beginning-timestamps of each word. Non-vocal sections are not explicitly annotated (but remain included in the last preceding word). We prefer to leave it this way, to enable comparison to previous work, evaluated on this dataset.
The audio has instrumental accompaniment. An example song can be seen [https://www.dropbox.com/sh/8pp4u2xg93z36d4/AAAsCE2eYW68gxRhKiPH_VvFa?dl=0 here].

You can read in detail about how the dataset was used for the first time here: [https://pdfs.semanticscholar.org/547d/7a5d105380562ca3543bf05b4d5f7a8bee66.pdf Integrating Additional Chord Information Into HMM-Based Lyrics-to-Audio Alignment]. The dataset has been kindly provided by Sungkyun Chang.

* file duration up to 5:40 minutes (total time: 1h 19m)
* 5050 words annotated in total

==== Gracenote Dataset ====
The dataset contains 8 pop music song excerpts with instrumental accompaniment, with annotations of beginning-timestamps of each word. The dataset has been used in the recent [https://ieeexplore.ieee.org/abstract/document/7952235/references paper].

* file duration up to 1:11 (total time: 11m)
* 1181 words annotated in total

=== Phonetization ===
A popular choice for phonetization of the words is the [http://www.speech.cs.cmu.edu/cgi-bin/cmudict CMU pronunciation dictionary]. One can phonetize them with the [http://www.speech.cs.cmu.edu/tools/lextool.html online tool]. A list of all words of both datasets, which are outside of the [https://github.com/georgid/AlignmentDuration/blob/noteOnsets/src/for_english/cmudict.0.6d.syll list of CMU words] is given [https://www.dropbox.com/s/flu4cpqff916bas/words_not_in_dict?dl=0 here].

=== Audio Format ===

The data are sound wav/mp3 files, plus the associated word boundaries (in csv-like .txt/.tsv files)

* CD-quality (PCM, 16-bit, 44100 Hz)
* single channel (mono) for a cappella and two channels for original

==Evaluation==

The submitted algorithms will be evaluated at the word boundaries for the originally mixed songs (a cappella singing + instrumental accompaniment). Evaluation metrics on the a cappella singing can be reported as well on request, for the sake of getting insights on the impact of instrumental accompaniment on the algorithm, but will not be considered for the ranking.

'''Average absolute error/deviation''' Initially utilized in [http://www.cs.tut.fi/~mesaros/pubs/autalign_cr.pdf Mesaros and Virtanen (2008)], the absolute error measures the time displacement between the actual timestamp and its estimate at the beginning and the end of each lyrical unit. The error is then averaged over all individual errors. An error in absolute terms has the drawback that the perception of an error with the same duration can be different depending on the tempo of the song.
Here is a [https://github.com/georgid/AlignmentEvaluation/blob/126c3fa5fa1994acdcfbe3ea1344acfe71ae2b8e/test/EvalMetricsTest.py#L117 test] of using this metric.

'''Percentage of correct segments''' The perceptual dependence on tempo is mitigated by measuring the percentage of the total length of the segments, labeled correctly to the total duration of the song. This metric is suggested by [https://www.researchgate.net/publication/224241940_LyricSynchronizer_Automatic_Synchronization_System_Between_Musical_Audio_Signals_and_Lyrics Fujihara et al. (2011), Figure 9].
Here is a [https://github.com/georgid/AlignmentEvaluation/blob/126c3fa5fa1994acdcfbe3ea1344acfe71ae2b8e/test/EvalMetricsTest.py#L76 test] of using this metric.

'''Percentage of correct estimates according to a tolerance window''' A metric that takes into consideration that the onset displacements from ground truth below a certain threshold could be tolerated by human listeners. We use 0.3 seconds as the tolerance window. This metric is suggested in [https://pdfs.semanticscholar.org/547d/7a5d105380562ca3543bf05b4d5f7a8bee66.pdf Integrating Additional Chord Information Into HMM-Based Lyrics-to-Audio Alignment].
Here is a [https://github.com/georgid/AlignmentEvaluation/blob/126c3fa5fa1994acdcfbe3ea1344acfe71ae2b8e/test/EvalMetricsTest.py#L151 test] of using this metric.

For more detailed definition and formulas about the metrics, please check the section 2.2.1 of [https://doi.org/10.5281/zenodo.841979 this thesis].

'''To obtain all three metrics for one detected output:'''

<code> python [https://github.com/georgid/AlignmentEvaluation/blob/master/align_eval/eval.py eval.py] <file path of the reference word boundaries> <file path of the detected word boundaries> </code>

Note that evaluation scripts depend on [https://github.com/craffel/mir_eval/ mir_eval].

== Submission Format ==

Every submission must be packed into a docker image containing the bash file <code>main.sh</code> in the root folder.

=== Input Data ===
Participating algorithms will have to receive the following input format:

* Audio in wav, 44.1kHz, stereo.
* Lyrics in .txt file where each word is separated by a space, each lyrics phrase is separated by a line break mark (\n).

=== Output File Format ===

The alignment output file format is a tab-delimited ASCII text format.

Three column text file of the format

<onset_time(sec)>\t<offset_time(sec)>\t<label>\n
<onset_time(sec)>\t<offset_time(sec)>\t<label>\n
...

where \t denotes a tab, \n denotes the end of the line. The < and > characters are not included. An example output file would look something like:

0.000 5.223 word1
5.223 15.101 word2
15.101 20.334 word3

'''NOTE:''' the offset timestamps column is utilized only by the percentage of correct segments metric. Therefore skipping the second column is acceptable, and could result in degraded performance of this respective metric only.

=== Command line calling format ===

The submitted algorithm must take as arguments .wav file, .txt file as well as the full output path and filename of the output file in the following format:

<pre>your_program %input_audio %input_txt %output_txt</pre>

=== Packaging submissions ===

Please see [[Submission Guidelines]] first if you are not familiar with docker.

* Every submission must be packed into a docker image
* Every submission will be deployed and evaluated automatically with <code>docker run</code>

Accepted submission form:

* Docker Hub: You can create a free account at [https://hub.docker.com/ Docker Hub] and upload your docker image there.
* Github Container Registry: If you are using Github, you can use the Github Container Registry to upload your docker image.
* Google Drive: You can upload your docker image to Google Drive and share the link with the evaluation organizers.

Notice that MIREX server currently does not support docker image upload. This might be supported in future years.

Here are the docker commands that will be used to run all systems:

<pre>
docker run -v dataset_folder:/app/data %input_audio %input_txt %output_txt
</pre>

== Time and hardware limits ==

A Linux server with one Nvidia GeForce RTX 3090 is used for evaluation. CPU, OS, and memory specifications will be announced later.

Time limit: within 5 times the total duration of the test set.

== Bibliography ==

Stoller, D. and Durand, S. and Ewert, S. (2019) End-to-end Lyrics Alignment for Polyphonic Music Using An Audio-to-Character Recognition Model. ICASSP 2019.

Sharma B, Gupta C. (2019) Automatic Lyrics-to-audio Alignment on Polyphonic Music Using Singing-adapted Acoustic Models. ICASSP 2019

Lee S. W., Scott, J. (2017) Word-level lyrics-audio synchronization using separated vocals", Acoustics Speech and Signal Processing, ICASSP IEEE International Conference on, pp. 646-650

Chang, S., & Lee, K. (2017). Lyrics-to-Audio Alignment by Unsupervised Discovery of Repetitive Patterns in Vowel Acoustics. arXiv preprint arXiv:1701.06078.

Pons, J. Gong, R. and Serra, X. (2017). Score-informed syllable segmentation for a cappella singing voice with convolutional neural networks. ISMIR 2017

Kruspe, A. (2016). Bootstrapping a System for Phoneme Recognition and Keyword Spotting in Unaccompanied Singing, ISMIR 2016

Dzhambazov, G. and Serra, X. (2015) Modeling of phoneme durations for alignment between polyphonic audio and lyrics, in 12th Sound and Music Computing Conference

Fujihara, H., & Goto, M. (2012). Lyrics-to-audio alignment and its application. In Dagstuhl Follow-Ups (Vol. 3). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.

Mauch, M., Fujihara, H., & Goto, M. (2012). Integrating additional chord information into HMM-based lyrics-to-audio alignment. IEEE Transactions on Audio, Speech, and Language Processing, 20(1), 200-210.

Fujihara, H. Goto, M. Ogata, J. and Okuno, H. G. (2011) Lyricsynchronizer: Automatic synchronization system between musical audio signals and lyrics, IEEE Journal of Selected Topics in Signal Processing

Mesaros, A. and Virtanen, T. (2008), Automatic alignment of music audio and lyrics, in Proceedings of the 11th Int. Conference on Digital Audio Effects (DAFx-08), Espoo, Finland, 2008.

2026:Audio Downbeat Estimation

2026-06-29T19:29:54Z

Junyan: /* Data */

== Description ==

Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation].

The aim of the automatic downbeat estimation task is to identify the locations of downbeats in a collection of sound files. While this is similar to the Audio Beat Tracking task, here the aim is to find the first beat of each bar (measure) rather than all beat times. Algorithms are '''not''' required to estimate beat times or time-signature in addition to downbeats.

Submitted algorithms will be evaluated in terms of their accuracy in finding downbeat locations (only) as annotated by musical experts across several diverse datasets.

== Data ==

=== Test Data ===

Participants are not allowed to use any split of the following datasets for training, validation, model selection, parameter tuning, or any other development purpose:

'''GTZAN''': The GTZAN dataset contains 1,000 30-second excerpts covering ten music genres. It provides a broad genre-balanced collection for evaluating beat and downbeat estimation systems beyond ballroom dance music.

'''Hidden test set''': We may include an additional hidden test set, mainly consisting of pop-style music. Details about this hidden set will be announced later if it is included in the evaluation.

=== Other Collections ===

Other collections may be used, but participants must clearly state their dataset usage and, when applicable, the data collection process in the extended abstract.

'''Ballroom'''
The Ballroom dataset contains eight different dance styles (Cha Cha, Jive, Quickstep, Rumba, Samba, Tango, Viennese Waltz and Waltz). It consists of 697 excerpts of 30s in duration. We removed duplicates from the dataset as suggested by [http://media.aau.dk/null_space_pursuits/2014/01/ballroom-dataset.html Bob Sturm], which finally yields '''685''' excerpts.
We are using the audio files available [http://mtg.upf.edu/ismir2004/contest/tempoContest/node5.html here] (see Gouyon et al. (2006)) and ground truth annotations available [https://github.com/CPJKU/BallroomAnnotations here] (see Krebs et al. (2013)).

'''HarmonixSet'''
912 pop songs with beat, downbeat and structure annotations ([https://github.com/urinieto/harmonixset/tree/main/dataset/beats_and_downbeats Annotation Files], [https://huggingface.co/datasets/m-a-p/harmonixset_bigvgan/tree/main Dataset]).

'''Turkish Data'''
The Turkish corpus is an extended version of the annotated data used in Srinivasamurthy et al. (2014). It includes '''82''' excerpts of one
minute length each, and each piece belongs to one of three
rhythm classes that are referred to as usul in Turkish Art
music. 32 pieces are in the 9/8-usul Aksak, 20 pieces
in the 10/8-usul Curcuna, 30 samples in the 8/8-usul
Düyek.

'''Cretan Data'''
The corpus of Cretan music consists of '''42''' full length pieces of Cretan leaping dances. While there are several dances that differ in terms of their steps, the differences in
the sound are most noticeable in the melodic content, and all pieces can be considered to belong to one rhythmic style. All these dances are usually notated using a 2/4 time signature,
and the accompanying rhythmical patterns are usually played on a Cretan lute. While a variety of rhythmic patterns exist, they do not relate to a specific dance and can be
assumed to occur in all of the 42 songs in this corpus.

'''Carnatic Data'''
The Carnatic music dataset is a subset of the CompMusic [http://compmusic.upf.edu/carnatic-rhythm-dataset Carnatic Music Rhythm Dataset]. It includes '''118''' two minute long excerpts spanning four most commonly used tālas (the rhythmic framework of Carnatic music, consisting of time cycles) of Carnatic music. There are 30 examples in each of ādi tāla (8 beats/cycle), rūpaka tāla (3 beats/cycle) and miśra chāpu tāla (7 beats/cycle), and 28 examples in khaṇḍa chāpu tāla (5 beats/cycle). The beats of the tāla in miśra chāpu and khaṇḍa chāpu are non-uniform, but for consistency with other datasets, a uniform beat pulse was obtained by interpolating the non-uniformly spaced beat locations. The recordings consist of both vocal and instrumental music recordings representative of the present day performance practice. All recordings contain percussion accompaniment, mainly the Mridangam.

'''HJDB'''
The HJDB dataset contains '''235''' excerpts of Hardcore, Jungle and Drum and Bass music between 30s and 2 minutes in length. All excerpts are in 4/4 and have a constant tempo.
For further information see Hockman et al (2012).

=== Audio Formats ===

The data are monophonic sound files

* CD-quality (PCM, 16-bit, 44100 Hz) for all except Ballroom (originally lower quality, but resampled to 44100 Hz)
* single channel (mono)

== Submission Format ==
Submissions to this task will have to conform to a specified format detailed below. Submissions should be packaged and contain at least two files: The algorithm itself and a README containing contact information and detailing, in full, the use of the algorithm.

=== Input Data ===
Participating algorithms will have to read audio in the following format:

* Sample rate: 44.1 KHz
* Sample size: 16 bit
* Number of channels: 1 (mono)
* Encoding: WAV

=== Output Data ===

The downbeat estimation algorithms will return downbeat times in an ASCII text file for each input .wav audio file. The specification of this output file is immediately below.

=== Output File Format (Audio Downbeat Estimation) ===

The downbeat output file format is an ASCII text format. Each downbeat time is specified, in seconds, on its own line. Specifically,

<downbeat time (in seconds)>\n

where \n denotes the end of line. The < and > characters are not included. An example output file would look something like:

0.243
1.486
2.729

=== Algorithm Calling Format ===

The submitted algorithm must take as arguments a SINGLE .wav file to perform the downbeat estimation as well as the full output path and filename of the output file. The ability to specify the output path and file name is essential. Denoting the input .wav file path and name as %input and the output file path and name as %output, a program called foobar could be called from the command-line as follows:

foobar %input %output
foobar -i %input -o %output

Moreover, if your submission takes additional parameters, such as a detection threshold, foobar could be called like:

foobar .1 %input %output
foobar -param1 .1 -i %input -o %output

If your submission is in MATLAB, it should be submitted as a function. Once again, the function must contain String inputs for the full path and names of the input and output files. Parameters could also be specified as input arguments of the function. For example:

foobar('%input','%output')
foobar(.1,'%input','%output')

=== README File ===

A README file accompanying each submission should contain explicit instructions on how to to run the program (as well as contact information, etc.). In particular, each command line to run should be specified, using %input for the input sound file and %output for the resulting text file.

For instance, to test the program foobar with different values for parameters param1, the README file would look like:

foobar -param1 .1 -i %input -o %output
foobar -param1 .15 -i %input -o %output
foobar -param1 .2 -i %input -o %output
foobar -param1 .25 -i %input -o %output
foobar -param1 .3 -i %input -o %output
...

For a submission using MATLAB, the README file could look like:

matlab -r "foobar(.1,'%input','%output');quit;"
matlab -r "foobar(.15,'%input','%output');quit;"
matlab -r "foobar(.2,'%input','%output');quit;"
matlab -r "foobar(.25,'%input','%output');quit;"
matlab -r "foobar(.3,'%input','%output');quit;"
...

The different command lines to evaluate the performance of each parameter set over the whole database will be generated automatically from each line in the README file containing both '%input' and '%output' strings.

== Evaluation Procedure ==

For the evaluation procedure we will use
*'''F-measure''' - the standard calculation as used in onset and beat tracking evaluation with a +/-70ms window, see Dixon (2007).

Given the high diversity of musical styles included in the task, results will be reported per each individual dataset.

== Time and hardware limits ==
Due to the potentially high number of participants in this and other audio tasks, limits on the runtime of submissions may be imposed. Specific details will be announced with the evaluation instructions.

= Bibliography =

S. Dixon, F. Gouyon and G. Widmer, [http://ismir2004.ismir.net/proceedings/p093-page-509-paper165.pdf Towards Characterisation of Music via Rhythmic Patterns], In Proceedings of the 5th International Conference on Music Information Retrieval (ISMIR 2004), pp 509-516.

S. Dixon, [http://www.eecs.qmul.ac.uk/~simond/pub/2007/jnmr07.pdf Evaluation of audio beat tracking system BeatRoot], Journal of New Music Research, vol. 36, no. 1, pp. 39-51, 2007.

J. A. Hockman, M. E. P. Davies, I. Fujinaga.[http://ismir2012.ismir.net/event/papers/169-ismir-2012.pdf ONE IN THE JUNGLE: Downbeat Detection in Hardcore, Jungle, and Drum and Bass], In Proceedings of 13th International Society for Music Information Retrieval Conference (ISMIR), Porto, Portugal pp. 169-174, 2012.

F. Krebs, S. Boeck, and G. Widmer, [http://www.cp.jku.at/research/papers/Krebs_etal_ISMIR_2013.pdf Rhythmic Pattern Modeling for Beat- and Downbeat Tracking in Musical Audio], In Proceedings of 14th International Society for Music Information Retrieval Conference (ISMIR), Curitiba, Brazil, 2013.

M. Mauch, C. Cannam, M. E. P. Davies, S. Dixon, C. Harte, S. Kolozali and D. Tidhar, [http://ismir2009.ismir.net/proceedings/LBD-18.pdf OMRAS2 Metadata Project 2009], Late-breaking session at the 10th International Conference on Music Information Retrieval, 2009.

A. Srinivasamurthy, A. Holzapfel, and Xavier Serra, [http://www.tandfonline.com/doi/full/10.1080/09298215.2013.879902 In Search of Automatic Rhythm Analysis Methods for Turkish and Indian Art Music], Journal of New Music Research, vol. 43, no. 1, pp. 94-114, 2014.

F. Gouyon, A. Klapuri, S. Dixon, M. Alonso, G. Tzanetakis, C. Uhle, and P. Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech and Language Processing 14(5), pp.1832-1844, 2006.

2026:Music Structure Analysis

2026-06-29T19:28:43Z

Junyan: Created page with "== Music Structure Analysis (MIREX 2026) == Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation]. === Description..."

== Music Structure Analysis (MIREX 2026) ==

Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation].

=== Description ===

The aim of the MIREX Music Structure Analysis task is to identify and label key structural sections in musical audio. Understanding the musical form (e.g., intro, verse, chorus) is fundamental to music understanding and a crucial component in many music information retrieval applications. While traditional approaches focused on segmenting music into internally consistent, but arbitrarily labeled, sections (e.g., A, B, C), this task has evolved.

Since 2020, a new paradigm has emerged, focusing on '''functional structure analysis'''. The goal is to segment the audio and assign a specific functional label to each segment from a predefined set of common musical functions. This task challenges systems to perform both accurate boundary detection and correct functional classification.

This task builds upon a history of structural segmentation evaluations, first run in MIREX 2009. Recent works driving this updated focus include:
* Wang, J. C., Hung, Y. N., & Smith, J. B. (2022, May). To catch a chorus, verse, intro, or anything else: Analyzing a song with structural functions. In ''ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)'' (pp. 416-420). IEEE.
* Kim, T., & Nam, J. (2023, October). All-in-one metrical and functional structure analysis with neighborhood attentions on demixed audio. In ''2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)'' (pp. 1-5). IEEE.
* Buisson, M., McFee, B., Essid, S., & Crayencour, H. C. (2024). Self-supervised learning of multi-level audio representations for music segmentation. ''IEEE/ACM Transactions on Audio, Speech, and Language Processing''.

For MIREX 2026, participants are required to segment musical audio and classify each segment into one of seven functional categories: '''‘intro’, ‘verse’, ‘chorus’, ‘bridge’, ‘inst’ (instrumental), ‘outro’, or ‘other’'''. The 'other' category can be used for segments that do not fit into the primary six functional labels or for non-musical content if explicitly defined by the dataset annotations being mapped.

=== Data ===

Other collections may be used, but participants must clearly state their dataset usage and, when applicable, the data collection process in the extended abstract. The Harmonix Set train and validation splits may also be used for training and development, but the Harmonix Set test split is held out.

We use the relabeled Harmonix Set for evaluation. The test split is held out for evaluation, and participants may use the train and validation splits for training models.

https://huggingface.co/datasets/m-a-p/harmonixset_bigvgan/tree/main

==== Collections ====
The evaluation will utilize datasets previously established in MIREX. Annotations from these diverse collections will be mapped to the seven target functional labels for consistent evaluation.
* '''The MIREX 2009 Collection''': 297 pieces, largely derived from the work of the Beatles.
* '''MIREX 2010 RWC collection''': 100 pieces of popular music. This collection has two sets of ground truths. The first was originally included with the RWC dataset. The second set provides segment boundary annotations (see [http://hal.inria.fr/docs/00/47/34/79/PDF/PI-1948.pdf Pechuho et al., 2010] for details).
* '''MIREX 2012 dataset''': Over 1,000 annotated pieces covering a range of musical styles, with the majority annotated by two independent annotators.

Participants should be aware that original labels in these datasets (e.g., 'verse1', 'solo', 'fade-out') will need to be mapped to the seven specified functional categories for evaluation. Guidelines for this mapping will be provided, or a standard mapping will be applied during evaluation.

==== Audio Formats (Input to Algorithms) ====
Algorithms should be prepared to process audio with the following characteristics:
* Sample rate: 44.1 kHz
* Bit depth: 16 bit
* Number of channels: 1 (mono)
* Encoding: WAV

=== Submission Format ===

Submissions will be handled via '''CodeBench'''. Participants are required to submit their results in a specific format, as detailed below. You will upload a single file containing the segmentation results for all test audio files.

==== Output Data Format ====
The output must be a '''list of dictionaries''' in a text-based format (e.g., JSON parsable). Each dictionary in the list corresponds to one audio file and must contain two keys: <tt>'id'</tt> (the identifier of the audio file, e.g., '1.wav') and <tt>'result'</tt> (a list of segment predictions). Each segment prediction is a list containing two elements: a two-element list with the <tt>[start_time, end_time]</tt> of the segment in seconds, and the <tt>label</tt> string for that segment.

The labels must be one of the seven target functional categories: <tt>'intro'</tt>, <tt>'verse'</tt>, <tt>'chorus'</tt>, <tt>'bridge'</tt>, <tt>'inst'</tt>, <tt>'outro'</tt>, <tt>'silence'</tt>.

Example of the content of the submitted file:
<pre>
[
{
'id': 'track01.wav',
'result': [
[[0.000, 15.500], 'intro'],
[[15.500, 45.230], 'verse'],
[[45.230, 75.800], 'chorus'],
[[75.800, 90.000], 'outro']
]
},
{
'id': 'track02.wav',
'result': [
[[0.000, 20.100], 'verse'],
[[20.100, 38.500], 'chorus'],
[[38.500, 55.000], 'verse'],
[[55.000, 72.600], 'chorus'],
[[72.600, 89.000], 'bridge'],
[[89.000, 105.000], 'chorus'],
[[105.000, 115.500], 'outro']
]
}
]
</pre>
Ensure that <tt>offset_time</tt> of one segment is the <tt>onset_time</tt> of the next segment, and segments cover the entire duration of the piece analyzed. The first segment must start at <tt>0.0</tt>.

=== Evaluation Procedures ===

Evaluation will focus on both the accuracy of the detected segment boundaries and the correctness of the assigned functional labels. The primary metrics are:

# '''Frame-Level Accuracy (ACC)''':
# Both the system output and the ground truth will be converted into time-series of labels at a fine temporal resolution (e.g., 10ms or 100ms frames). Accuracy is calculated as the proportion of frames that are correctly labeled by the system compared to the ground truth across the entire dataset. This metric evaluates the overall correctness of segment labels and their temporal extents.

# '''Boundary Retrieval Hit Rate F-Measures (HR.5F and HR3F)''':
# This metric assesses the system's ability to correctly identify segment boundaries.
# * A predicted boundary is considered a '''hit''' if it falls within a certain tolerance window of a ground truth boundary.
# * Two tolerance windows will be used:
# ** 0.5 seconds: For finer precision.
# ** 3.0 seconds: For coarser, more perceptually relevant boundaries.
# * Based on these hits, '''Precision (P)''', '''Recall (R)''', and '''F-measure (F1-score)''' will be calculated for boundary detection at both tolerance levels.
# <math>P = \frac{\text{Number of correctly retrieved boundaries}}{\text{Total number of retrieved boundaries}}</math>
# <math>R = \frac{\text{Number of correctly retrieved boundaries}}{\text{Total number of ground truth boundaries}}</math>
# <math>F = \frac{2 \times P \times R}{P + R}</math>
# * The reported metrics will be '''HR.5F''' (F-measure with 0.5s tolerance) and '''HR3F''' (F-measure with 3s tolerance).

==== Baseline ====
The performance of the method described in '''Kim, T., & Nam, J. (2023). All-in-one metrical and functional structure analysis with neighborhood attentions on demixed audio.''' will serve as a baseline for this task. Participants are encouraged to develop systems that surpass this baseline.

=== Relevant Development Collections ===
While the MIREX datasets will be used for evaluation, participants may find the following publicly available annotated corpora useful for development. Please note that the annotations in these corpora will also need to be mapped to the 7-class functional labeling scheme if used for training models for this task.

* Jouni Paulus's [http://www.cs.tut.fi/sgn/arg/paulus/structure.html structure analysis page] links to a corpus of 177 Beatles songs ([http://www.cs.tut.fi/sgn/arg/paulus/beatles_sections_TUT.zip zip file]). The TUTstructure07 dataset, containing 557 songs, is also listed [http://www.cs.tut.fi/sgn/arg/paulus/TUTstructure07_files.html here].
* Ewald Peiszer's [http://www.ifs.tuwien.ac.at/mir/audiosegmentation.html thesis page] links to a portion of his corpus: 43 non-Beatles pop songs (including 10 J-pop songs) ([http://www.ifs.tuwien.ac.at/mir/audiosegmentation/dl/ep_groundtruth_excl_Paulus.zip zip file]).

These public corpora offer over 200 songs that can be adapted for development purposes.

=== Time and Hardware Limits ===
Due to the nature of the CodeBench platform and the potentially high number of participants, limits on the runtime and computational resources for submissions may be imposed. Specific details regarding these limits will be provided closer to the submission deadline. A general guideline is that analysis should be computationally feasible. For reference, a hard limit of '''24 hours''' for total analysis time over the evaluation dataset was imposed in previous iterations, and a similar constraint might apply.

2026:New Task Proposals

2026-06-29T19:27:53Z

Junyan: /* Continued Tasks */

These are the accepted task proposals for MIREX 2026.

==Task Descriptions==

Accepted challenge proposals
* [[2026:Audio-to-Score Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Audio Instrument Recognition]] <TC: [mailto:wenye.ma@mail.mcgill.ca Wenye Ma]>
* [[2026:Rhythm Game Chart Generation]] <TC: [mailto:ziyun.liu@univ-lille.fr Ziyun Liu] & [mailto:carolina.carusi@kaist.ac.kr Carolina Carusi]>
* [[2026:Music Evaluation via CMI-RewardBench]] <TC: [mailto:yinghao.ma@qmul.ac.uk Yinghao Ma], Yonghyun Kim, Haiwen Xia, Junwon Lee>
* [[2026:AI-Generated Music Detection]] <TC: [mailto:ldzhangyx@outlook.com Yixiao Zhang] & [mailto:you.zhang@rochester.edu Neil Zhang]>
* [[2026:Music Performance Difficulty Prediction]] <TC: [mailto:huan.zhang@qmul.ac.uk Huan Zhang] & [mailto:pedro@songscription.ai Pedro Ramoneda]>

Accepted task captain proposals
* [[2026:Lyrics Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>

Task captains may edit the names before Jul 1.

==Continued Tasks==

Modern MIR Tasks
* [[2026:Symbolic Music Generation]]

Standardized Evaluation Suites
* [[2026:Audio Chord Estimation]]
* [[2026:Audio Beat Tracking]]
* [[2026:Audio Key Detection]]
* [[2026:Audio Downbeat Estimation]]
* [[2026:Music Structure Analysis]]
* [[2026:Lyrics-to-Audio Alignment]]

2026:Audio Downbeat Estimation

2026-06-29T19:27:22Z

Junyan: Created page with "== Description == Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation]. The aim of the automatic downbeat estimati..."

== Description ==

Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation].

The aim of the automatic downbeat estimation task is to identify the locations of downbeats in a collection of sound files. While this is similar to the Audio Beat Tracking task, here the aim is to find the first beat of each bar (measure) rather than all beat times. Algorithms are '''not''' required to estimate beat times or time-signature in addition to downbeats.

Submitted algorithms will be evaluated in terms of their accuracy in finding downbeat locations (only) as annotated by musical experts across several diverse datasets.

== Data ==

=== Test Data ===

Participants are not allowed to use any split of the following datasets for training, validation, model selection, parameter tuning, or any other development purpose:

'''GTZAN''': The GTZAN dataset contains 1,000 30-second excerpts covering ten music genres. It provides a broad genre-balanced collection for evaluating beat and downbeat estimation systems beyond ballroom dance music.

'''Hidden test set''': We may include an additional hidden test set, mainly consisting of pop-style music. Details about this hidden set will be announced later if it is included in the evaluation.

=== Other Collections ===

Other collections may be used, but participants must clearly state their dataset usage and, when applicable, the data collection process in the extended abstract.

'''Ballroom'''
The Ballroom dataset contains eight different dance styles (Cha Cha, Jive, Quickstep, Rumba, Samba, Tango, Viennese Waltz and Waltz). It consists of 697 excerpts of 30s in duration. We removed duplicates from the dataset as suggested by [http://media.aau.dk/null_space_pursuits/2014/01/ballroom-dataset.html Bob Sturm], which finally yields '''685''' excerpts.
We are using the audio files available [http://mtg.upf.edu/ismir2004/contest/tempoContest/node5.html here] (see Gouyon et al. (2006)) and ground truth annotations available [https://github.com/CPJKU/BallroomAnnotations here] (see Krebs et al. (2013)).

'''HarmonixSet'''
912 pop songs with beat, downbeat and structure annotations (https://github.com/urinieto/harmonixset/tree/main/dataset/beats_and_downbeats).

'''Turkish Data'''
The Turkish corpus is an extended version of the annotated data used in Srinivasamurthy et al. (2014). It includes '''82''' excerpts of one
minute length each, and each piece belongs to one of three
rhythm classes that are referred to as usul in Turkish Art
music. 32 pieces are in the 9/8-usul Aksak, 20 pieces
in the 10/8-usul Curcuna, 30 samples in the 8/8-usul
Düyek.

'''Cretan Data'''
The corpus of Cretan music consists of '''42''' full length pieces of Cretan leaping dances. While there are several dances that differ in terms of their steps, the differences in
the sound are most noticeable in the melodic content, and all pieces can be considered to belong to one rhythmic style. All these dances are usually notated using a 2/4 time signature,
and the accompanying rhythmical patterns are usually played on a Cretan lute. While a variety of rhythmic patterns exist, they do not relate to a specific dance and can be
assumed to occur in all of the 42 songs in this corpus.

'''Carnatic Data'''
The Carnatic music dataset is a subset of the CompMusic [http://compmusic.upf.edu/carnatic-rhythm-dataset Carnatic Music Rhythm Dataset]. It includes '''118''' two minute long excerpts spanning four most commonly used tālas (the rhythmic framework of Carnatic music, consisting of time cycles) of Carnatic music. There are 30 examples in each of ādi tāla (8 beats/cycle), rūpaka tāla (3 beats/cycle) and miśra chāpu tāla (7 beats/cycle), and 28 examples in khaṇḍa chāpu tāla (5 beats/cycle). The beats of the tāla in miśra chāpu and khaṇḍa chāpu are non-uniform, but for consistency with other datasets, a uniform beat pulse was obtained by interpolating the non-uniformly spaced beat locations. The recordings consist of both vocal and instrumental music recordings representative of the present day performance practice. All recordings contain percussion accompaniment, mainly the Mridangam.

'''HJDB'''
The HJDB dataset contains '''235''' excerpts of Hardcore, Jungle and Drum and Bass music between 30s and 2 minutes in length. All excerpts are in 4/4 and have a constant tempo.
For further information see Hockman et al (2012).

=== Audio Formats ===

The data are monophonic sound files

* CD-quality (PCM, 16-bit, 44100 Hz) for all except Ballroom (originally lower quality, but resampled to 44100 Hz)
* single channel (mono)

== Submission Format ==
Submissions to this task will have to conform to a specified format detailed below. Submissions should be packaged and contain at least two files: The algorithm itself and a README containing contact information and detailing, in full, the use of the algorithm.

=== Input Data ===
Participating algorithms will have to read audio in the following format:

* Sample rate: 44.1 KHz
* Sample size: 16 bit
* Number of channels: 1 (mono)
* Encoding: WAV

=== Output Data ===

The downbeat estimation algorithms will return downbeat times in an ASCII text file for each input .wav audio file. The specification of this output file is immediately below.

=== Output File Format (Audio Downbeat Estimation) ===

The downbeat output file format is an ASCII text format. Each downbeat time is specified, in seconds, on its own line. Specifically,

<downbeat time (in seconds)>\n

where \n denotes the end of line. The < and > characters are not included. An example output file would look something like:

0.243
1.486
2.729

=== Algorithm Calling Format ===

The submitted algorithm must take as arguments a SINGLE .wav file to perform the downbeat estimation as well as the full output path and filename of the output file. The ability to specify the output path and file name is essential. Denoting the input .wav file path and name as %input and the output file path and name as %output, a program called foobar could be called from the command-line as follows:

foobar %input %output
foobar -i %input -o %output

Moreover, if your submission takes additional parameters, such as a detection threshold, foobar could be called like:

foobar .1 %input %output
foobar -param1 .1 -i %input -o %output

If your submission is in MATLAB, it should be submitted as a function. Once again, the function must contain String inputs for the full path and names of the input and output files. Parameters could also be specified as input arguments of the function. For example:

foobar('%input','%output')
foobar(.1,'%input','%output')

=== README File ===

A README file accompanying each submission should contain explicit instructions on how to to run the program (as well as contact information, etc.). In particular, each command line to run should be specified, using %input for the input sound file and %output for the resulting text file.

For instance, to test the program foobar with different values for parameters param1, the README file would look like:

foobar -param1 .1 -i %input -o %output
foobar -param1 .15 -i %input -o %output
foobar -param1 .2 -i %input -o %output
foobar -param1 .25 -i %input -o %output
foobar -param1 .3 -i %input -o %output
...

For a submission using MATLAB, the README file could look like:

matlab -r "foobar(.1,'%input','%output');quit;"
matlab -r "foobar(.15,'%input','%output');quit;"
matlab -r "foobar(.2,'%input','%output');quit;"
matlab -r "foobar(.25,'%input','%output');quit;"
matlab -r "foobar(.3,'%input','%output');quit;"
...

The different command lines to evaluate the performance of each parameter set over the whole database will be generated automatically from each line in the README file containing both '%input' and '%output' strings.

== Evaluation Procedure ==

For the evaluation procedure we will use
*'''F-measure''' - the standard calculation as used in onset and beat tracking evaluation with a +/-70ms window, see Dixon (2007).

Given the high diversity of musical styles included in the task, results will be reported per each individual dataset.

== Time and hardware limits ==
Due to the potentially high number of participants in this and other audio tasks, limits on the runtime of submissions may be imposed. Specific details will be announced with the evaluation instructions.

= Bibliography =

S. Dixon, F. Gouyon and G. Widmer, [http://ismir2004.ismir.net/proceedings/p093-page-509-paper165.pdf Towards Characterisation of Music via Rhythmic Patterns], In Proceedings of the 5th International Conference on Music Information Retrieval (ISMIR 2004), pp 509-516.

S. Dixon, [http://www.eecs.qmul.ac.uk/~simond/pub/2007/jnmr07.pdf Evaluation of audio beat tracking system BeatRoot], Journal of New Music Research, vol. 36, no. 1, pp. 39-51, 2007.

J. A. Hockman, M. E. P. Davies, I. Fujinaga.[http://ismir2012.ismir.net/event/papers/169-ismir-2012.pdf ONE IN THE JUNGLE: Downbeat Detection in Hardcore, Jungle, and Drum and Bass], In Proceedings of 13th International Society for Music Information Retrieval Conference (ISMIR), Porto, Portugal pp. 169-174, 2012.

F. Krebs, S. Boeck, and G. Widmer, [http://www.cp.jku.at/research/papers/Krebs_etal_ISMIR_2013.pdf Rhythmic Pattern Modeling for Beat- and Downbeat Tracking in Musical Audio], In Proceedings of 14th International Society for Music Information Retrieval Conference (ISMIR), Curitiba, Brazil, 2013.

M. Mauch, C. Cannam, M. E. P. Davies, S. Dixon, C. Harte, S. Kolozali and D. Tidhar, [http://ismir2009.ismir.net/proceedings/LBD-18.pdf OMRAS2 Metadata Project 2009], Late-breaking session at the 10th International Conference on Music Information Retrieval, 2009.

A. Srinivasamurthy, A. Holzapfel, and Xavier Serra, [http://www.tandfonline.com/doi/full/10.1080/09298215.2013.879902 In Search of Automatic Rhythm Analysis Methods for Turkish and Indian Art Music], Journal of New Music Research, vol. 43, no. 1, pp. 94-114, 2014.

F. Gouyon, A. Klapuri, S. Dixon, M. Alonso, G. Tzanetakis, C. Uhle, and P. Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech and Language Processing 14(5), pp.1832-1844, 2006.

2026:Audio Key Detection

2026-06-29T19:20:01Z

Junyan: Created page with "==Description== Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation]. Audio Key Detection aims to identify the mus..."

==Description==

Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation].

Audio Key Detection aims to identify the musical key (e.g., C major, A minor) of an audio recording. This involves determining both the tonic (root pitch) and the mode (major or minor) from the audio signal.

== Data ==

=== Test Data ===

Participants are not allowed to use any split of the following datasets for training, validation, model selection, parameter tuning, or any other development purpose:

'''GiantSteps Key''': The GiantSteps Key Dataset comprises 604 two-minute excerpts of electronic dance music tracks, primarily sourced from Beatport. Each excerpt is annotated with one of 24 musical keys—12 major and 12 minor—providing a standardized benchmark for evaluating key classification performance.

=== Other Collections ===

Other collections may be used, but participants must clearly state their dataset usage and, when applicable, the data collection process in the extended abstract.

==== GiantSteps MTG Keys ====

1,077 additional songs that extend the GiantSteps Key dataset. NOTICE: You may use GiantSteps MTG Keys for training, but should exclude the songs from the original GiantSteps Key collection. Link: https://github.com/GiantSteps/giantsteps-mtg-key-dataset/tree/master/annotations/key

==== Isophonics Dataset ====
The Isophonics Dataset includes 225 songs from artists such as The Beatles, Queen, and Zweieck. Each track is annotated with key, chord, beat, and structural segmentation information, offering a comprehensive resource for evaluating key detection algorithms across a diverse range of popular music.

== Evaluation Procedures ==
The error analysis will center on comparing the key identified by the algorithm to the actual key of the piece. The key of the piece is the one defined by the composer in the title of the piece. We will then determine how "close" each identified key is to the corresponding correct key. Keys will be considered as "close" if they have one of the following relationships: distance of perfect fifth, relative major and minor, and parallel major and minor. A correct key assignment will be given a full point, and incorrect assignments will be allocated fractions of a point according to the following table:

{|border="1"
|'''Relation to Correct Key''' ||'''Points'''
|-
|Same||1.0
|-
|Perfect fifth||0.5
|-
|Relative major/minor||0.3
|-
|Parallel major/minor||0.2
|-
|Other||0.0
|}

The points are counted over all files and averaged. The number of correctly identified keys as well as the distribution of the errors is also reported.

== Submission Format ==

Submissions to this task will have to conform to a specified format detailed below. Submissions should be packaged and contain at least two files: The algorithm itself and a README containing contact information and detailing, in full, the use of the algorithm.

=== Input Data ===
Participating algorithms will have to read audio in the following format:

* Sample rate: 44.1 KHz
* Sample size: 16 bit
* Number of channels: 1 (mono)
* Encoding: WAV

=== Output Data ===

The audio key detection algorithms will return the estimated key in an individual ASCII text file for each input .wav audio file. The specification of this output file is immediately below.

=== Output File Format (Audio Key Detection) ===

The Audio Key Detection output file format is a single-line tab-delimited ASCII text format. The tonic is reported, followed by a TAB and the mode. For sharps, the "#" symbol is used (e.g. A# for A sharp), for flats, a lowercase "b" is used, e.g. (Bb for B flat). Therefore, the output file should be of the form:

<tonic {A, A#, Bb, ...}>\t<mode {major, minor}>\n

where \t denotes a tab, \n denotes the end of line. The < and > characters are not included. An example output file would look something like:

C major

or

G# minor

=== Algorithm Calling Format ===

The submitted algorithm must take as arguments a SINGLE .wav file to perform the melody extraction on as well as the full output path and filename of the output file. The ability to specify the output path and file name is essential. Denoting the input .wav file path and name as %input and the output file path and name as %output, a program called foobar could be called from the command-line as follows:

foobar %input %output
foobar -i %input -o %output

Moreover, if your submission takes additional parameters, foobar could be called like:

foobar .1 %input %output
foobar -param1 .1 -i %input -o %output

If your submission is in MATLAB, it should be submitted as a function. Once again, the function must contain String inputs for the full path and names of the input and output files. Parameters could also be specified as input arguments of the function. For example:

foobar('%input','%output')
foobar(.1,'%input','%output')

=== Packaging submissions ===

All submissions should include a README file including the following the information:

* Command line calling format for all executables including examples
* Number of threads/cores used or whether this should be specified on the command line
* Expected memory footprint
* Expected runtime
* Approximately how much scratch disk space will the submission need to store any feature/cache files?
* Any required environments/architectures (and versions) such as Matlab, Java, Python, Bash, Ruby etc.
* Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

==== README File ====

A README file accompanying each submission should contain explicit instructions on how to to run the program (as well as contact information, etc.). In particular, each command line to run should be specified, using %input for the input sound file and %output for the resulting text file.

For instance, to test the program foobar with a specific value for parameter param1, the README file would look like:

foobar -param1 .1 -i %input -o %output

2026:Audio Beat Tracking

2026-06-29T19:12:01Z

== Description ==

Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation].

The aim of the automatic beat tracking task is to track each beat locations in a collection of sound files. Unlike the Audio Tempo Extraction task, which aim is to detect tempi for each file, the beat tracking task aims at detecting all beat locations in recordings. The algorithms will be evaluated in terms of their accuracy in predicting beat locations annotated by a group of listeners.

== Data ==

=== Test Data ===

Participants are not allowed to use any split of the following datasets for training, validation, model selection, parameter tuning, or any other development purpose:

'''SMC Dataset''': The SMC dataset contains challenging excerpts with beat annotations, designed to test beat tracking systems on material with ambiguous, expressive, or otherwise difficult rhythmic content.

'''GTZAN dataset''': The GTZAN dataset contains 1,000 30-second excerpts covering ten music genres. It provides a broad genre-balanced collection for evaluating beat tracking systems beyond rhythmically regular dance music.

=== Other Collections ===

Other collections may be used, but participants must clearly state their dataset usage and, when applicable, the data collection process in the extended abstract.

==== Ballroom Dataset ====
The Ballroom Dataset contains 685 short excerpts (30 seconds each) from various ballroom dance genres, such as Waltz, Tango, Rumba, and Jive. Each excerpt includes annotated beat and meter information, making the dataset ideal for evaluating beat tracking and tempo estimation methods in rhythmically regular dance music.

==== Hainsworth Dataset ====
The Hainsworth Dataset offers 222 music excerpts covering a wide range of genres, including classical, jazz, and popular music. Annotations include both beat and downbeat locations, providing a varied testing ground for assessing the performance of beat tracking systems across different musical styles.

== Submission Format ==
Submissions to this task will have to conform to a specified format detailed below. Submissions should be packaged and contain at least two files: The algorithm itself and a README containing contact information and detailing, in full, the use of the algorithm.

=== Input Data ===
Participating algorithms will have to read audio in the following format:

* Sample rate: 44.1 KHz
* Sample size: 16 bit
* Number of channels: 1 (mono)
* Encoding: WAV

=== Output Data ===

The beat tracking algorithms will return beat-times in an ASCII text file for each input .wav audio file. The specification of this output file is immediately below.

=== Output File Format (Audio Beat tracking) ===

The Beat Tracking output file format is an ASCII text format. Each beat time is specified, in seconds, on its own line. Specifically,

<beat time(in seconds)>\n

where \n denotes the end of line. The < and > characters are not included. An example output file would look something like:

0.243
0.486
0.729

=== Algorithm Calling Format ===

The submitted algorithm must take as arguments a SINGLE .wav file to perform the onset detection on as well as the full output path and filename of the output file. The ability to specify the output path and file name is essential. Denoting the input .wav file path and name as %input and the output file path and name as %output, a program called foobar could be called from the command-line as follows:

foobar %input %output
foobar -i %input -o %output

Moreover, if your submission takes additional parameters, such as a detection threshold, foobar could be called like:

foobar .1 %input %output
foobar -param1 .1 -i %input -o %output

If your submission is in MATLAB, it should be submitted as a function. Once again, the function must contain String inputs for the full path and names of the input and output files. Parameters could also be specified as input arguments of the function. For example:

foobar('%input','%output')
foobar(.1,'%input','%output')

=== README File ===

A README file accompanying each submission should contain explicit instructions on how to to run the program (as well as contact information, etc.). In particular, each command line to run should be specified, using %input for the input sound file and %output for the resulting text file.

For instance, to test the program foobar with different values for parameters param1, the README file would look like:

foobar -param1 .1 -i %input -o %output
foobar -param1 .15 -i %input -o %output
foobar -param1 .2 -i %input -o %output
foobar -param1 .25 -i %input -o %output
foobar -param1 .3 -i %input -o %output
...

For a submission using MATLAB, the README file could look like:

matlab -r "foobar(.1,'%input','%output');quit;"
matlab -r "foobar(.15,'%input','%output');quit;"
matlab -r "foobar(.2,'%input','%output');quit;"
matlab -r "foobar(.25,'%input','%output');quit;"
matlab -r "foobar(.3,'%input','%output');quit;"
...

The different command lines to evaluate the performance of each parameter set over the whole database will be generated automatically from each line in the README file containing both '%input' and '%output' strings.

== Evaluation Procedures ==

The evaluation methods are taken from the beat evaluation toolbox and
are described in the following technical report:

M. E. P. Davies, N. Degara and M. D. Plumbley. "Evaluation methods for musical audio beat tracking algorithms". [http://www.elec.qmul.ac.uk/people/markp/2009/DaviesDegaraPlumbley09-evaluation-tr.pdf ''Technical Report C4DM-TR-09-06'']. This link now works! :)

For further details on the specifics of the methods please refer to the
paper. However, here is a brief summary with appropriate references:

*'''F-measure''' - the standard calculation as used in onset evaluation but
with a 70ms window.

S. Dixon, "Onset detection revisited," in ''Proceedings of 9th
International Conference on Digital Audio Effects (DAFx)'', Montreal,
Canada, pp. 133-137, 2006.

S. Dixon, "Evaluation of audio beat tracking system beatroot," ''Journal
of New Music Research'', vol. 36, no. 1, pp. 39-51, 2007.

*'''Cemgil''' - beat accuracy is calculated using a Gaussian error function
with 40ms standard deviation.

A. T. Cemgil, B. Kappen, P. Desain, and H. Honing, "On tempo tracking:
Tempogram representation and Kalman filtering," ''Journal Of New Music
Research'', vol. 28, no. 4, pp. 259-273, 2001

*'''Goto''' - binary decision of correct or incorrect tracking based on
statistical properties of a beat error sequence.

M. Goto and Y. Muraoka, "Issues in evaluating beat tracking systems," in
''Working Notes of the IJCAI-97 Workshop on Issues in AI and Music -
Evaluation and Assessment'', 1997, pp. 9-16.

*'''PScore''' - McKinney's impulse train cross-correlation method as used in
2006.

M. F. McKinney, D. Moelants, M. E. P. Davies, and A. Klapuri,
"Evaluation of audio beat tracking and music tempo extraction
algorithms," ''Journal of New Music Research'', vol. 36, no. 1, pp. 1-16,
2007.

*'''CMLc''', '''CMLt''', '''AMLc''', '''AMLt''' - continuity-based evaluation methods based on
the longest continuously correctly tracked section.

S. Hainsworth, "Techniques for the automated analysis of musical audio,"
Ph.D. dissertation, Department of Engineering, Cambridge University,
2004.

A. P. Klapuri, A. Eronen, and J. Astola, "Analysis of the meter of
acoustic musical signals," IEEE Transactions on Audio, Speech and
Language Processing, vol. 14, no. 1, pp. 342-355, 2006.

*'''D''', '''Dg''' - information based criteria based on analysis of a beat error
histogram (note the results are measured in 'bits' and not percentages),
see the technical report for a description.

2026:Audio Chord Estimation

2026-06-29T16:27:15Z

Junyan: /* Other Collections */

= Description =

Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation].

This task requires participants to extract or transcribe a sequence of chords from an audio music recording. For many applications in music information retrieval, extracting the harmonic structure of an audio track is very desirable, for example for segmenting pieces into characteristic segments, for finding similar pieces, or for semantic analysis of music. The extraction of the harmonic structure requires the estimation of a sequence of chords that is as precise as possible. This includes the full characterisation of chords – root, quality, and bass note – as well as their chronological order, including specific onset times and durations. Audio chord estimation has a long history in MIREX, and readers interested in this history, especially with respect to evaluation methodology, should review the work of Christopher Harte (2010), Pauwels and Peeters (2013), and the [https://www.music-ir.org/mirex/wiki/The_Utrecht_Agreement_on_Chord_Evaluation “Utrecht Agreement”] on evaluation metrics. For python evaluation code, please refer to [https://craffel.github.io/mir_eval/#module-mir_eval.chord “mir_eval”].

= Data =

== Test Data ==

Participants are not allowed to use any split of the following datasets for training, validation, model selection, parameter tuning, or any other development purpose:

* Billboard 2013

We might include additional hidden test sets for the task.

== Other Collections ==

Other collections may be used, but participants must clearly state their dataset usage and, when applicable, the data collection process in the extended abstract.

The following datasets are commonly used for development or background comparison in audio chord estimation. They are listed here for context.

; Isophonics
: The collected Beatles, Queen, and Zweieck datasets from the Centre for Digital Music at Queen Mary, University of London (http://www.isophonics.net/), as used for Audio Chord Estimation in MIREX for many years. Available from http://www.isophonics.net/. See also Matthias Mauch’s dissertation (2010) and Harte et al.’s introductory paper (2005).
; Billboard 2012
: An abridged version of the ''Billboard'' dataset from McGill University, including a representative sample of American popular music from the 1950s through the 1990s. Available from http://billboard.music.mcgill.ca. See also Ashley Burgoyne’s dissertation (2012) and Burgoyne et al.’s introductory paper (2011). Parsing tools for the data are available from http://hackage.haskell.org/package/billboard-parser/ and documented by De Haas and Burgoyne (2012).

== Training and Testing ==

The ground-truth files contain one line per unique chord, in the form <code>{start_time end_time chord}</code>, e.g.,
<pre>...
41.2631021 44.2456460 B:maj
44.2456460 45.7201230 E:maj
45.7201230 47.2061900 E:7/3
47.2061900 48.6922670 A:maj
48.6922670 50.1551240 A:min/b3
...</pre>
Start and end times are in seconds from the start of the file. Chord labels follow the syntax proposed by C. Harte et al. (2005). Please note that the syntax has changed slightly since since it was originally described; in particular, the root is no longer implied as a voiced element of a chord so a C major chord (notes C, E and G) should be written C:(1,3,5) instead of just C:(3,5) if using the interval list representation. As before, the labels C and C:maj are equivalent to C:(1,3,5).

= Evaluation =

To evaluate the quality of an automatic transcription, a transcription is compared to ground truth created by one or more human annotators. MIREX typically uses ''chord symbol recall'' (CSR) to estimate how well the predicted chords match the ground truth:

<math>\textrm{CSR} = \frac{\text{total duration of segments where annotation equals estimation}} {\text{total duration of annotated segments}}</math>

In previous years, MIREX has used an approximate CSR calculated by sampling both the ground-truth and the automatic annotations every 10 ms and dividing the number of correctly annotated samples by the total number of samples. Following Christopher Harte (2010, §8.1.2), however, we can view the ground-truth and estimated annotations as continuous segmentations of the audio and calculate the CSR by considering the cumulative length of the correctly overlapping segments. This way of calculating the CSR is more precise, as the precision of the frame-based method is limited by the frame length, and computationally more efficient, as it reduces the number of segment comparisons. Because pieces of music come in a wide variety of lengths, we will weight the CSR by the length of the song when computing an average for a given corpus. This final number is referred to as the ''weighted chord symbol recall'' (WCSR).

== Chord Vocabularies ==

We propose a set of single chord evaluation measures for MIREX that extends the previous iterations of MIREX and combines it with evaluation measures proposed in the literature, providing a more complete assessment of the transcription quality. Following Pauwels and Peeters (2013), we suggest using the CSR with five different chord vocabulary mappings.

In each of these calculations, the full chord descriptions of either the estimated or the ground-truth transcriptions, which might contain complex chord annotations, would be mapped to the following classes:

# Chord root note only;
# Major and minor: {<code>N, maj, min</code>};
# Seventh chords: {<code>N, maj, min, maj7, min7, 7</code>};
# Major and minor with inversions: {<code>N, maj, min, maj/3, min/b3, maj/5, min/5</code>}; or
# Seventh chords with inversions: {<code>N, maj, min, maj7, min7, 7, maj/3, min/b3, maj7/3, min7/b3, 7/3, maj/5, min/5, maj7/5, min7/5, 7/5, maj7/7, min7/b7, 7/b7</code>}.

With the exception of no-chords, calculating the vocabulary mapping involves examining the root note, the bass note, and the relative interval structure of the chord labels. A mapping exists if both the root notes and bass notes match, and the structure of the output label is the largest possible subset of the input label given the vocabulary. For instance, in the major and minor case, <code>G:7(#9)</code> is mapped to <code>G:maj</code> because the interval set of <code>G:maj</code>, {<code>1,3,5</code>}, is a subset of the interval set of the <code>G:7(#9)</code>, {<code>1,3,5,b7,#9</code>}. In the seventh-chord case, <code>G:7(#9)</code> is mapped to <code>G:7</code> instead because the interval set of <code>G:7</code> {<code>1, 3, 5, b7</code>} is also a subset of <code>G:7(#9)</code> but is larger than <code>G:maj</code>. If a chord cannot be represented by a certain class, e.g., mapping a <code>D:aug</code> or <code>F:sus4(9)</code> to {<code>maj, min</code>}, the chord is excluded from the evaluation if it occurs in the ground-truth, and it is considered a mismatch if it occurs in an estimated annotation.

{|
|+ Most frequent chord qualities in the McGill ''Billboard'' corpus.
! Quality
! Freq. (%)
! Cum. Freq (%)
|-
|maj
|52
|52
|-
|min
|13
|65
|-
|7
|10
|75
|-
|min7
|8
|83
|-
|maj7
|3
|86
|-
|5
|2
|88
|-
|1
|2
|90
|-
|maj(9)
|1
|91
|-
|maj6
|1
|92
|-
|sus4
|1
|93
|-
|sus7
|1
|94
|-
|sus9
|1
|94
|-
|7(#9)
|1
|95
|-
|min9
|1
|96
|}

Our recommendations are motivated by the frequencies of chord qualities in the ''Billboard'' corpus (see table above), which is a balanced sample of American popular music from the 1950s through the 1990s (J.A. Burgoyne, Wild, and Fujinaga 2011). Pure major and minor chords alone account for 65 percent of all chords encountered, whereas augmented and diminished triads account for 0.2 percent or less of the corpus each. Our arguments for our particular seventh-chord vocabulary as opposed to the set of all tetrads follows similar reasoning; our proposed vocabulary accounts for 86 percent of all chords, whereas no other standard type of seventh chord accounts for more than 0.2 percent of the corpus. In future years, the table suggests that we might consider introducing vocabularies including power chords, and possibly suspended chords or added sixths and ninths as well.

== Chord Segmentation ==

Besides CSR, the chord transcription literature includes several other metrics for evaluating chord transcriptions, which mainly focus on the segmentation of the automatic transcription. We propose to include the directional Hamming distance in the evaluation. The directional Hamming distance is calculated by finding for each annotated segment the maximally overlapping segment in the other annotation, and then summing the differences ((S. A. Abdallah et al. 2005); (Mauch 2010, §2.3.3)). Depending on the order of application, the directional Hamming distance yields a measure of over- or under segmentation. Both directions can be combined to yield an overall quality metric (Christopher Harte 2010, §8.3.2):

<math>Q = 1 - \frac{\text{maximum of directional Hamming distances in either direction}} {\text{total duration of song}}</math>

= Submission Format =

== Audio Format ==

Audio tracks in the training directory will be encoded as 44.1 kHz 16bit mono WAV files.

== I/O Format ==

The algorithms should output text files with a similar format to that used in the ground truth transcriptions. That is to say, they should be flat text files with chord segment labels and times arranged thus:

<pre>start_time end_time chord_label</pre>
with elements separated by white spaces, times given in seconds, chord labels corresponding to the syntax described by C. Harte et al. (2005), and one chord segment per line. As in all benchmarks after 2008, end times are a mandatory component of the output. For the evaluation process we will assume enharmonic equivalence for chord roots. We will no longer accept participants who would only like to be evaluated on major/minor chords and want to use the number format.

== Command line calling format ==

Submissions using machine learning models must also submit their trained models. Training on the evaluation server is no longer supported starting from this year. We will execute the following commands for testing:

<pre>
your_program prepare
your_program do_chord_identification "/path/to/input1.wav" "/path/to/output1.wav.txt"
your_program do_chord_identification "/path/to/input2.wav" "/path/to/output2.wav.txt"
...
</pre>

In the results directory, there should be one file for each testfile with same name as the test file + <code>.txt</code>. Programs can use the folder <code>/app/temp</code> if they need to keep temporary cache files or to download pretrained models. Standard output and standard error will be logged.

No internet access is allowed during the inference stage (<code>do_chord_identification</code>). Please contact us if your model requires internet access (e.g., model API call) during inference.

== Packaging submissions ==

* You can directly upload your submission (up to 5GB) to the MIREX submission site. If you need more storage you may contact the evaluation organizers.

=== Non-docker submission ===

* We recommend the participants submit a docker image to ensure that the evaluation team can easily run it.
* Non-docker submissions should contain a README file to include the setup procedure.

=== Docker Submission ===

* A docker submission will be deployed and evaluated automatically with <code>docker run</code>

Here are the docker commands that will be used to evaluate all systems:

<pre>
docker run -v temp_folder:/app/temp your_image_name prepare
docker run -v temp_folder:/app/temp -v dataset_folder:/app/data do_chord_identification "/app/data/input1.wav" "/app/data/output1.wav.txt"
docker run -v temp_folder:/app/temp -v dataset_folder:/app/data do_chord_identification "/app/data/input2.wav" "/app/data/output2.wav.txt"
...
</pre>

==== Example project ====

A sample project containing code and <code>Dockerfile</code> for a simple audio chord estimation baseline can be found in [https://github.com/futuremirex/audio_chord_estimation_sample_project https://github.com/futuremirex/audio_chord_estimation_sample_project].

Notice that this is not a compiled docker image - you need to build the image by your own before submitting it.

= Time and Hardware limits =

A Linux server with one Nvidia GeForce RTX 3090 is used for evaluation. CPU, OS, and memory specifications will be announced later.

Time limit: within 5 times the total duration of the test set.

= Bibliography =

Abdallah, Samer A., Katy Noland, Mark B. Sandler, Michael Casey, and Christophe Rhodes. 2005. “Theory and Evaluation of a Bayesian Music Structure Extractor.” In ''Proceedings of the International Society for Music Information Retrieval Conference'', 420–425.

Burgoyne, J. A., J. Wild, and I. Fujinaga. 2011. “An expert ground truth set for audio chord recognition and music analysis.” In ''Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR)'', 633–638.

Burgoyne, John Ashley. 2012. “Stochastic Processes and Database-Driven Musicology.” Ph.D. diss. Montréal, Québec, Canada: McGill University.

Haas, W. B. de, and John~Ashley Burgoyne. 2012. ''Parsing the Billboard Chord Transcriptions''. Technical report UU-CS- 2012-018, Department of Information and Computing Sciences, Utrecht University.

Harte, C., M. Sandler, S. Abdallah, and E. Gómez. 2005. “Symbolic representation of musical chords: A proposed syntax for text annotations.” In ''Proceedings of the 6th International Society for Music Information Retrieval Conference (ISMIR)'', 66–71.

Harte, Christopher. 2010. “Towards automatic extraction of harmony information from music signals.” Ph.D. diss. Queen Mary, University of London.

Mauch, Matthias. 2010. “Automatic Chord Transcription from Audio Using Computational Models of Musical Context.” Ph.D. diss. Queen Mary University of London.

Pauwels, Johan, and Geoffroy Peeters. 2013. “Evaluating automatically estimated chord sequences.” In ''Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)''. Vancouver, British Columbia, Canada.

2026:Audio Chord Estimation

2026-06-29T16:26:23Z

Junyan: /* Data */

= Description =

Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation].

This task requires participants to extract or transcribe a sequence of chords from an audio music recording. For many applications in music information retrieval, extracting the harmonic structure of an audio track is very desirable, for example for segmenting pieces into characteristic segments, for finding similar pieces, or for semantic analysis of music. The extraction of the harmonic structure requires the estimation of a sequence of chords that is as precise as possible. This includes the full characterisation of chords – root, quality, and bass note – as well as their chronological order, including specific onset times and durations. Audio chord estimation has a long history in MIREX, and readers interested in this history, especially with respect to evaluation methodology, should review the work of Christopher Harte (2010), Pauwels and Peeters (2013), and the [https://www.music-ir.org/mirex/wiki/The_Utrecht_Agreement_on_Chord_Evaluation “Utrecht Agreement”] on evaluation metrics. For python evaluation code, please refer to [https://craffel.github.io/mir_eval/#module-mir_eval.chord “mir_eval”].

= Data =

== Test Data ==

Participants are not allowed to use any split of the following datasets for training, validation, model selection, parameter tuning, or any other development purpose:

* Billboard 2013

We might include additional hidden test sets for the task.

== Other Collections ==

Other collections may be used, but participants must clearly state their dataset usage and, when applicable, the data collection process in the extended abstract.

The following datasets are commonly used for development or background comparison in audio chord estimation. They are listed here for context; participants remain responsible for checking that any collections they use do not overlap with the held-out datasets.

; Isophonics
: The collected Beatles, Queen, and Zweieck datasets from the Centre for Digital Music at Queen Mary, University of London (http://www.isophonics.net/), as used for Audio Chord Estimation in MIREX for many years. Available from http://www.isophonics.net/. See also Matthias Mauch’s dissertation (2010) and Harte et al.’s introductory paper (2005).
; Billboard 2012
: An abridged version of the ''Billboard'' dataset from McGill University, including a representative sample of American popular music from the 1950s through the 1990s. Available from http://billboard.music.mcgill.ca. See also Ashley Burgoyne’s dissertation (2012) and Burgoyne et al.’s introductory paper (2011). Parsing tools for the data are available from http://hackage.haskell.org/package/billboard-parser/ and documented by De Haas and Burgoyne (2012).

== Training and Testing ==

The ground-truth files contain one line per unique chord, in the form <code>{start_time end_time chord}</code>, e.g.,
<pre>...
41.2631021 44.2456460 B:maj
44.2456460 45.7201230 E:maj
45.7201230 47.2061900 E:7/3
47.2061900 48.6922670 A:maj
48.6922670 50.1551240 A:min/b3
...</pre>
Start and end times are in seconds from the start of the file. Chord labels follow the syntax proposed by C. Harte et al. (2005). Please note that the syntax has changed slightly since since it was originally described; in particular, the root is no longer implied as a voiced element of a chord so a C major chord (notes C, E and G) should be written C:(1,3,5) instead of just C:(3,5) if using the interval list representation. As before, the labels C and C:maj are equivalent to C:(1,3,5).

= Evaluation =

To evaluate the quality of an automatic transcription, a transcription is compared to ground truth created by one or more human annotators. MIREX typically uses ''chord symbol recall'' (CSR) to estimate how well the predicted chords match the ground truth:

<math>\textrm{CSR} = \frac{\text{total duration of segments where annotation equals estimation}} {\text{total duration of annotated segments}}</math>

In previous years, MIREX has used an approximate CSR calculated by sampling both the ground-truth and the automatic annotations every 10 ms and dividing the number of correctly annotated samples by the total number of samples. Following Christopher Harte (2010, §8.1.2), however, we can view the ground-truth and estimated annotations as continuous segmentations of the audio and calculate the CSR by considering the cumulative length of the correctly overlapping segments. This way of calculating the CSR is more precise, as the precision of the frame-based method is limited by the frame length, and computationally more efficient, as it reduces the number of segment comparisons. Because pieces of music come in a wide variety of lengths, we will weight the CSR by the length of the song when computing an average for a given corpus. This final number is referred to as the ''weighted chord symbol recall'' (WCSR).

== Chord Vocabularies ==

We propose a set of single chord evaluation measures for MIREX that extends the previous iterations of MIREX and combines it with evaluation measures proposed in the literature, providing a more complete assessment of the transcription quality. Following Pauwels and Peeters (2013), we suggest using the CSR with five different chord vocabulary mappings.

In each of these calculations, the full chord descriptions of either the estimated or the ground-truth transcriptions, which might contain complex chord annotations, would be mapped to the following classes:

# Chord root note only;
# Major and minor: {<code>N, maj, min</code>};
# Seventh chords: {<code>N, maj, min, maj7, min7, 7</code>};
# Major and minor with inversions: {<code>N, maj, min, maj/3, min/b3, maj/5, min/5</code>}; or
# Seventh chords with inversions: {<code>N, maj, min, maj7, min7, 7, maj/3, min/b3, maj7/3, min7/b3, 7/3, maj/5, min/5, maj7/5, min7/5, 7/5, maj7/7, min7/b7, 7/b7</code>}.

With the exception of no-chords, calculating the vocabulary mapping involves examining the root note, the bass note, and the relative interval structure of the chord labels. A mapping exists if both the root notes and bass notes match, and the structure of the output label is the largest possible subset of the input label given the vocabulary. For instance, in the major and minor case, <code>G:7(#9)</code> is mapped to <code>G:maj</code> because the interval set of <code>G:maj</code>, {<code>1,3,5</code>}, is a subset of the interval set of the <code>G:7(#9)</code>, {<code>1,3,5,b7,#9</code>}. In the seventh-chord case, <code>G:7(#9)</code> is mapped to <code>G:7</code> instead because the interval set of <code>G:7</code> {<code>1, 3, 5, b7</code>} is also a subset of <code>G:7(#9)</code> but is larger than <code>G:maj</code>. If a chord cannot be represented by a certain class, e.g., mapping a <code>D:aug</code> or <code>F:sus4(9)</code> to {<code>maj, min</code>}, the chord is excluded from the evaluation if it occurs in the ground-truth, and it is considered a mismatch if it occurs in an estimated annotation.

{|
|+ Most frequent chord qualities in the McGill ''Billboard'' corpus.
! Quality
! Freq. (%)
! Cum. Freq (%)
|-
|maj
|52
|52
|-
|min
|13
|65
|-
|7
|10
|75
|-
|min7
|8
|83
|-
|maj7
|3
|86
|-
|5
|2
|88
|-
|1
|2
|90
|-
|maj(9)
|1
|91
|-
|maj6
|1
|92
|-
|sus4
|1
|93
|-
|sus7
|1
|94
|-
|sus9
|1
|94
|-
|7(#9)
|1
|95
|-
|min9
|1
|96
|}

Our recommendations are motivated by the frequencies of chord qualities in the ''Billboard'' corpus (see table above), which is a balanced sample of American popular music from the 1950s through the 1990s (J.A. Burgoyne, Wild, and Fujinaga 2011). Pure major and minor chords alone account for 65 percent of all chords encountered, whereas augmented and diminished triads account for 0.2 percent or less of the corpus each. Our arguments for our particular seventh-chord vocabulary as opposed to the set of all tetrads follows similar reasoning; our proposed vocabulary accounts for 86 percent of all chords, whereas no other standard type of seventh chord accounts for more than 0.2 percent of the corpus. In future years, the table suggests that we might consider introducing vocabularies including power chords, and possibly suspended chords or added sixths and ninths as well.

== Chord Segmentation ==

Besides CSR, the chord transcription literature includes several other metrics for evaluating chord transcriptions, which mainly focus on the segmentation of the automatic transcription. We propose to include the directional Hamming distance in the evaluation. The directional Hamming distance is calculated by finding for each annotated segment the maximally overlapping segment in the other annotation, and then summing the differences ((S. A. Abdallah et al. 2005); (Mauch 2010, §2.3.3)). Depending on the order of application, the directional Hamming distance yields a measure of over- or under segmentation. Both directions can be combined to yield an overall quality metric (Christopher Harte 2010, §8.3.2):

<math>Q = 1 - \frac{\text{maximum of directional Hamming distances in either direction}} {\text{total duration of song}}</math>

= Submission Format =

== Audio Format ==

Audio tracks in the training directory will be encoded as 44.1 kHz 16bit mono WAV files.

== I/O Format ==

The algorithms should output text files with a similar format to that used in the ground truth transcriptions. That is to say, they should be flat text files with chord segment labels and times arranged thus:

<pre>start_time end_time chord_label</pre>
with elements separated by white spaces, times given in seconds, chord labels corresponding to the syntax described by C. Harte et al. (2005), and one chord segment per line. As in all benchmarks after 2008, end times are a mandatory component of the output. For the evaluation process we will assume enharmonic equivalence for chord roots. We will no longer accept participants who would only like to be evaluated on major/minor chords and want to use the number format.

== Command line calling format ==

Submissions using machine learning models must also submit their trained models. Training on the evaluation server is no longer supported starting from this year. We will execute the following commands for testing:

<pre>
your_program prepare
your_program do_chord_identification "/path/to/input1.wav" "/path/to/output1.wav.txt"
your_program do_chord_identification "/path/to/input2.wav" "/path/to/output2.wav.txt"
...
</pre>

In the results directory, there should be one file for each testfile with same name as the test file + <code>.txt</code>. Programs can use the folder <code>/app/temp</code> if they need to keep temporary cache files or to download pretrained models. Standard output and standard error will be logged.

No internet access is allowed during the inference stage (<code>do_chord_identification</code>). Please contact us if your model requires internet access (e.g., model API call) during inference.

== Packaging submissions ==

* You can directly upload your submission (up to 5GB) to the MIREX submission site. If you need more storage you may contact the evaluation organizers.

=== Non-docker submission ===

* We recommend the participants submit a docker image to ensure that the evaluation team can easily run it.
* Non-docker submissions should contain a README file to include the setup procedure.

=== Docker Submission ===

* A docker submission will be deployed and evaluated automatically with <code>docker run</code>

Here are the docker commands that will be used to evaluate all systems:

<pre>
docker run -v temp_folder:/app/temp your_image_name prepare
docker run -v temp_folder:/app/temp -v dataset_folder:/app/data do_chord_identification "/app/data/input1.wav" "/app/data/output1.wav.txt"
docker run -v temp_folder:/app/temp -v dataset_folder:/app/data do_chord_identification "/app/data/input2.wav" "/app/data/output2.wav.txt"
...
</pre>

==== Example project ====

A sample project containing code and <code>Dockerfile</code> for a simple audio chord estimation baseline can be found in [https://github.com/futuremirex/audio_chord_estimation_sample_project https://github.com/futuremirex/audio_chord_estimation_sample_project].

Notice that this is not a compiled docker image - you need to build the image by your own before submitting it.

= Time and Hardware limits =

A Linux server with one Nvidia GeForce RTX 3090 is used for evaluation. CPU, OS, and memory specifications will be announced later.

Time limit: within 5 times the total duration of the test set.

= Bibliography =

Abdallah, Samer A., Katy Noland, Mark B. Sandler, Michael Casey, and Christophe Rhodes. 2005. “Theory and Evaluation of a Bayesian Music Structure Extractor.” In ''Proceedings of the International Society for Music Information Retrieval Conference'', 420–425.

Burgoyne, J. A., J. Wild, and I. Fujinaga. 2011. “An expert ground truth set for audio chord recognition and music analysis.” In ''Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR)'', 633–638.

Burgoyne, John Ashley. 2012. “Stochastic Processes and Database-Driven Musicology.” Ph.D. diss. Montréal, Québec, Canada: McGill University.

Haas, W. B. de, and John~Ashley Burgoyne. 2012. ''Parsing the Billboard Chord Transcriptions''. Technical report UU-CS- 2012-018, Department of Information and Computing Sciences, Utrecht University.

Harte, C., M. Sandler, S. Abdallah, and E. Gómez. 2005. “Symbolic representation of musical chords: A proposed syntax for text annotations.” In ''Proceedings of the 6th International Society for Music Information Retrieval Conference (ISMIR)'', 66–71.

Harte, Christopher. 2010. “Towards automatic extraction of harmony information from music signals.” Ph.D. diss. Queen Mary, University of London.

Mauch, Matthias. 2010. “Automatic Chord Transcription from Audio Using Computational Models of Musical Context.” Ph.D. diss. Queen Mary University of London.

Pauwels, Johan, and Geoffroy Peeters. 2013. “Evaluating automatically estimated chord sequences.” In ''Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)''. Vancouver, British Columbia, Canada.

2026:Audio Chord Estimation

2026-06-29T16:25:12Z

Junyan: /* Data */

= Description =

Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation].

This task requires participants to extract or transcribe a sequence of chords from an audio music recording. For many applications in music information retrieval, extracting the harmonic structure of an audio track is very desirable, for example for segmenting pieces into characteristic segments, for finding similar pieces, or for semantic analysis of music. The extraction of the harmonic structure requires the estimation of a sequence of chords that is as precise as possible. This includes the full characterisation of chords – root, quality, and bass note – as well as their chronological order, including specific onset times and durations. Audio chord estimation has a long history in MIREX, and readers interested in this history, especially with respect to evaluation methodology, should review the work of Christopher Harte (2010), Pauwels and Peeters (2013), and the [https://www.music-ir.org/mirex/wiki/The_Utrecht_Agreement_on_Chord_Evaluation “Utrecht Agreement”] on evaluation metrics. For python evaluation code, please refer to [https://craffel.github.io/mir_eval/#module-mir_eval.chord “mir_eval”].

= Data =

== Held-out Datasets ==

Participants are not allowed to use any split of the following datasets for training, validation, model selection, parameter tuning, or any other development purpose:

* Billboard 2013

We might include additional hidden test sets for the task.

== Other Collections ==

Other collections may be used, but participants must clearly state their dataset usage and, when applicable, the data collection process in the extended abstract.

The following datasets are commonly used for development or background comparison in audio chord estimation. They are listed here for context; participants remain responsible for checking that any collections they use do not overlap with the held-out datasets.

; Isophonics
: The collected Beatles, Queen, and Zweieck datasets from the Centre for Digital Music at Queen Mary, University of London (http://www.isophonics.net/), as used for Audio Chord Estimation in MIREX for many years. Available from http://www.isophonics.net/. See also Matthias Mauch’s dissertation (2010) and Harte et al.’s introductory paper (2005).
; Billboard 2012
: An abridged version of the ''Billboard'' dataset from McGill University, including a representative sample of American popular music from the 1950s through the 1990s. Available from http://billboard.music.mcgill.ca. See also Ashley Burgoyne’s dissertation (2012) and Burgoyne et al.’s introductory paper (2011). Parsing tools for the data are available from http://hackage.haskell.org/package/billboard-parser/ and documented by De Haas and Burgoyne (2012).

== Training and Testing ==

The ground-truth files contain one line per unique chord, in the form <code>{start_time end_time chord}</code>, e.g.,
<pre>...
41.2631021 44.2456460 B:maj
44.2456460 45.7201230 E:maj
45.7201230 47.2061900 E:7/3
47.2061900 48.6922670 A:maj
48.6922670 50.1551240 A:min/b3
...</pre>
Start and end times are in seconds from the start of the file. Chord labels follow the syntax proposed by C. Harte et al. (2005). Please note that the syntax has changed slightly since since it was originally described; in particular, the root is no longer implied as a voiced element of a chord so a C major chord (notes C, E and G) should be written C:(1,3,5) instead of just C:(3,5) if using the interval list representation. As before, the labels C and C:maj are equivalent to C:(1,3,5).

= Evaluation =

To evaluate the quality of an automatic transcription, a transcription is compared to ground truth created by one or more human annotators. MIREX typically uses ''chord symbol recall'' (CSR) to estimate how well the predicted chords match the ground truth:

<math>\textrm{CSR} = \frac{\text{total duration of segments where annotation equals estimation}} {\text{total duration of annotated segments}}</math>

In previous years, MIREX has used an approximate CSR calculated by sampling both the ground-truth and the automatic annotations every 10 ms and dividing the number of correctly annotated samples by the total number of samples. Following Christopher Harte (2010, §8.1.2), however, we can view the ground-truth and estimated annotations as continuous segmentations of the audio and calculate the CSR by considering the cumulative length of the correctly overlapping segments. This way of calculating the CSR is more precise, as the precision of the frame-based method is limited by the frame length, and computationally more efficient, as it reduces the number of segment comparisons. Because pieces of music come in a wide variety of lengths, we will weight the CSR by the length of the song when computing an average for a given corpus. This final number is referred to as the ''weighted chord symbol recall'' (WCSR).

== Chord Vocabularies ==

We propose a set of single chord evaluation measures for MIREX that extends the previous iterations of MIREX and combines it with evaluation measures proposed in the literature, providing a more complete assessment of the transcription quality. Following Pauwels and Peeters (2013), we suggest using the CSR with five different chord vocabulary mappings.

In each of these calculations, the full chord descriptions of either the estimated or the ground-truth transcriptions, which might contain complex chord annotations, would be mapped to the following classes:

# Chord root note only;
# Major and minor: {<code>N, maj, min</code>};
# Seventh chords: {<code>N, maj, min, maj7, min7, 7</code>};
# Major and minor with inversions: {<code>N, maj, min, maj/3, min/b3, maj/5, min/5</code>}; or
# Seventh chords with inversions: {<code>N, maj, min, maj7, min7, 7, maj/3, min/b3, maj7/3, min7/b3, 7/3, maj/5, min/5, maj7/5, min7/5, 7/5, maj7/7, min7/b7, 7/b7</code>}.

With the exception of no-chords, calculating the vocabulary mapping involves examining the root note, the bass note, and the relative interval structure of the chord labels. A mapping exists if both the root notes and bass notes match, and the structure of the output label is the largest possible subset of the input label given the vocabulary. For instance, in the major and minor case, <code>G:7(#9)</code> is mapped to <code>G:maj</code> because the interval set of <code>G:maj</code>, {<code>1,3,5</code>}, is a subset of the interval set of the <code>G:7(#9)</code>, {<code>1,3,5,b7,#9</code>}. In the seventh-chord case, <code>G:7(#9)</code> is mapped to <code>G:7</code> instead because the interval set of <code>G:7</code> {<code>1, 3, 5, b7</code>} is also a subset of <code>G:7(#9)</code> but is larger than <code>G:maj</code>. If a chord cannot be represented by a certain class, e.g., mapping a <code>D:aug</code> or <code>F:sus4(9)</code> to {<code>maj, min</code>}, the chord is excluded from the evaluation if it occurs in the ground-truth, and it is considered a mismatch if it occurs in an estimated annotation.

{|
|+ Most frequent chord qualities in the McGill ''Billboard'' corpus.
! Quality
! Freq. (%)
! Cum. Freq (%)
|-
|maj
|52
|52
|-
|min
|13
|65
|-
|7
|10
|75
|-
|min7
|8
|83
|-
|maj7
|3
|86
|-
|5
|2
|88
|-
|1
|2
|90
|-
|maj(9)
|1
|91
|-
|maj6
|1
|92
|-
|sus4
|1
|93
|-
|sus7
|1
|94
|-
|sus9
|1
|94
|-
|7(#9)
|1
|95
|-
|min9
|1
|96
|}

Our recommendations are motivated by the frequencies of chord qualities in the ''Billboard'' corpus (see table above), which is a balanced sample of American popular music from the 1950s through the 1990s (J.A. Burgoyne, Wild, and Fujinaga 2011). Pure major and minor chords alone account for 65 percent of all chords encountered, whereas augmented and diminished triads account for 0.2 percent or less of the corpus each. Our arguments for our particular seventh-chord vocabulary as opposed to the set of all tetrads follows similar reasoning; our proposed vocabulary accounts for 86 percent of all chords, whereas no other standard type of seventh chord accounts for more than 0.2 percent of the corpus. In future years, the table suggests that we might consider introducing vocabularies including power chords, and possibly suspended chords or added sixths and ninths as well.

== Chord Segmentation ==

Besides CSR, the chord transcription literature includes several other metrics for evaluating chord transcriptions, which mainly focus on the segmentation of the automatic transcription. We propose to include the directional Hamming distance in the evaluation. The directional Hamming distance is calculated by finding for each annotated segment the maximally overlapping segment in the other annotation, and then summing the differences ((S. A. Abdallah et al. 2005); (Mauch 2010, §2.3.3)). Depending on the order of application, the directional Hamming distance yields a measure of over- or under segmentation. Both directions can be combined to yield an overall quality metric (Christopher Harte 2010, §8.3.2):

<math>Q = 1 - \frac{\text{maximum of directional Hamming distances in either direction}} {\text{total duration of song}}</math>

= Submission Format =

== Audio Format ==

Audio tracks in the training directory will be encoded as 44.1 kHz 16bit mono WAV files.

== I/O Format ==

The algorithms should output text files with a similar format to that used in the ground truth transcriptions. That is to say, they should be flat text files with chord segment labels and times arranged thus:

<pre>start_time end_time chord_label</pre>
with elements separated by white spaces, times given in seconds, chord labels corresponding to the syntax described by C. Harte et al. (2005), and one chord segment per line. As in all benchmarks after 2008, end times are a mandatory component of the output. For the evaluation process we will assume enharmonic equivalence for chord roots. We will no longer accept participants who would only like to be evaluated on major/minor chords and want to use the number format.

== Command line calling format ==

Submissions using machine learning models must also submit their trained models. Training on the evaluation server is no longer supported starting from this year. We will execute the following commands for testing:

<pre>
your_program prepare
your_program do_chord_identification "/path/to/input1.wav" "/path/to/output1.wav.txt"
your_program do_chord_identification "/path/to/input2.wav" "/path/to/output2.wav.txt"
...
</pre>

In the results directory, there should be one file for each testfile with same name as the test file + <code>.txt</code>. Programs can use the folder <code>/app/temp</code> if they need to keep temporary cache files or to download pretrained models. Standard output and standard error will be logged.

No internet access is allowed during the inference stage (<code>do_chord_identification</code>). Please contact us if your model requires internet access (e.g., model API call) during inference.

== Packaging submissions ==

* You can directly upload your submission (up to 5GB) to the MIREX submission site. If you need more storage you may contact the evaluation organizers.

=== Non-docker submission ===

* We recommend the participants submit a docker image to ensure that the evaluation team can easily run it.
* Non-docker submissions should contain a README file to include the setup procedure.

=== Docker Submission ===

* A docker submission will be deployed and evaluated automatically with <code>docker run</code>

Here are the docker commands that will be used to evaluate all systems:

<pre>
docker run -v temp_folder:/app/temp your_image_name prepare
docker run -v temp_folder:/app/temp -v dataset_folder:/app/data do_chord_identification "/app/data/input1.wav" "/app/data/output1.wav.txt"
docker run -v temp_folder:/app/temp -v dataset_folder:/app/data do_chord_identification "/app/data/input2.wav" "/app/data/output2.wav.txt"
...
</pre>

==== Example project ====

A sample project containing code and <code>Dockerfile</code> for a simple audio chord estimation baseline can be found in [https://github.com/futuremirex/audio_chord_estimation_sample_project https://github.com/futuremirex/audio_chord_estimation_sample_project].

Notice that this is not a compiled docker image - you need to build the image by your own before submitting it.

= Time and Hardware limits =

A Linux server with one Nvidia GeForce RTX 3090 is used for evaluation. CPU, OS, and memory specifications will be announced later.

Time limit: within 5 times the total duration of the test set.

= Bibliography =

Abdallah, Samer A., Katy Noland, Mark B. Sandler, Michael Casey, and Christophe Rhodes. 2005. “Theory and Evaluation of a Bayesian Music Structure Extractor.” In ''Proceedings of the International Society for Music Information Retrieval Conference'', 420–425.

Burgoyne, J. A., J. Wild, and I. Fujinaga. 2011. “An expert ground truth set for audio chord recognition and music analysis.” In ''Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR)'', 633–638.

Burgoyne, John Ashley. 2012. “Stochastic Processes and Database-Driven Musicology.” Ph.D. diss. Montréal, Québec, Canada: McGill University.

Haas, W. B. de, and John~Ashley Burgoyne. 2012. ''Parsing the Billboard Chord Transcriptions''. Technical report UU-CS- 2012-018, Department of Information and Computing Sciences, Utrecht University.

Harte, C., M. Sandler, S. Abdallah, and E. Gómez. 2005. “Symbolic representation of musical chords: A proposed syntax for text annotations.” In ''Proceedings of the 6th International Society for Music Information Retrieval Conference (ISMIR)'', 66–71.

Harte, Christopher. 2010. “Towards automatic extraction of harmony information from music signals.” Ph.D. diss. Queen Mary, University of London.

Mauch, Matthias. 2010. “Automatic Chord Transcription from Audio Using Computational Models of Musical Context.” Ph.D. diss. Queen Mary University of London.

Pauwels, Johan, and Geoffroy Peeters. 2013. “Evaluating automatically estimated chord sequences.” In ''Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)''. Vancouver, British Columbia, Canada.

2026:Audio Chord Estimation

2026-06-29T16:24:58Z

Junyan: Created page with "= Description = Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation]. This task requires participants to extract o..."

= Description =

Standardized Evaluation Suites: [https://github.com/ismir-mirex/mirex-evaluation ismir-mirex/mirex-evaluation].

This task requires participants to extract or transcribe a sequence of chords from an audio music recording. For many applications in music information retrieval, extracting the harmonic structure of an audio track is very desirable, for example for segmenting pieces into characteristic segments, for finding similar pieces, or for semantic analysis of music. The extraction of the harmonic structure requires the estimation of a sequence of chords that is as precise as possible. This includes the full characterisation of chords – root, quality, and bass note – as well as their chronological order, including specific onset times and durations. Audio chord estimation has a long history in MIREX, and readers interested in this history, especially with respect to evaluation methodology, should review the work of Christopher Harte (2010), Pauwels and Peeters (2013), and the [https://www.music-ir.org/mirex/wiki/The_Utrecht_Agreement_on_Chord_Evaluation “Utrecht Agreement”] on evaluation metrics. For python evaluation code, please refer to [https://craffel.github.io/mir_eval/#module-mir_eval.chord “mir_eval”].

= Data =

== Held-out Datasets ==

Participants are not allowed to use any split of the following datasets for training, validation, model selection, parameter tuning, or any other development purpose:

* Billboard 2013

== Other Collections ==

Other collections may be used, but participants must clearly state their dataset usage and, when applicable, the data collection process in the extended abstract.

The following datasets are commonly used for development or background comparison in audio chord estimation. They are listed here for context; participants remain responsible for checking that any collections they use do not overlap with the held-out datasets.

We might include additional hidden test sets for the task.

; Isophonics
: The collected Beatles, Queen, and Zweieck datasets from the Centre for Digital Music at Queen Mary, University of London (http://www.isophonics.net/), as used for Audio Chord Estimation in MIREX for many years. Available from http://www.isophonics.net/. See also Matthias Mauch’s dissertation (2010) and Harte et al.’s introductory paper (2005).
; Billboard 2012
: An abridged version of the ''Billboard'' dataset from McGill University, including a representative sample of American popular music from the 1950s through the 1990s. Available from http://billboard.music.mcgill.ca. See also Ashley Burgoyne’s dissertation (2012) and Burgoyne et al.’s introductory paper (2011). Parsing tools for the data are available from http://hackage.haskell.org/package/billboard-parser/ and documented by De Haas and Burgoyne (2012).

== Training and Testing ==

The ground-truth files contain one line per unique chord, in the form <code>{start_time end_time chord}</code>, e.g.,
<pre>...
41.2631021 44.2456460 B:maj
44.2456460 45.7201230 E:maj
45.7201230 47.2061900 E:7/3
47.2061900 48.6922670 A:maj
48.6922670 50.1551240 A:min/b3
...</pre>
Start and end times are in seconds from the start of the file. Chord labels follow the syntax proposed by C. Harte et al. (2005). Please note that the syntax has changed slightly since since it was originally described; in particular, the root is no longer implied as a voiced element of a chord so a C major chord (notes C, E and G) should be written C:(1,3,5) instead of just C:(3,5) if using the interval list representation. As before, the labels C and C:maj are equivalent to C:(1,3,5).

= Evaluation =

To evaluate the quality of an automatic transcription, a transcription is compared to ground truth created by one or more human annotators. MIREX typically uses ''chord symbol recall'' (CSR) to estimate how well the predicted chords match the ground truth:

<math>\textrm{CSR} = \frac{\text{total duration of segments where annotation equals estimation}} {\text{total duration of annotated segments}}</math>

In previous years, MIREX has used an approximate CSR calculated by sampling both the ground-truth and the automatic annotations every 10 ms and dividing the number of correctly annotated samples by the total number of samples. Following Christopher Harte (2010, §8.1.2), however, we can view the ground-truth and estimated annotations as continuous segmentations of the audio and calculate the CSR by considering the cumulative length of the correctly overlapping segments. This way of calculating the CSR is more precise, as the precision of the frame-based method is limited by the frame length, and computationally more efficient, as it reduces the number of segment comparisons. Because pieces of music come in a wide variety of lengths, we will weight the CSR by the length of the song when computing an average for a given corpus. This final number is referred to as the ''weighted chord symbol recall'' (WCSR).

== Chord Vocabularies ==

We propose a set of single chord evaluation measures for MIREX that extends the previous iterations of MIREX and combines it with evaluation measures proposed in the literature, providing a more complete assessment of the transcription quality. Following Pauwels and Peeters (2013), we suggest using the CSR with five different chord vocabulary mappings.

In each of these calculations, the full chord descriptions of either the estimated or the ground-truth transcriptions, which might contain complex chord annotations, would be mapped to the following classes:

# Chord root note only;
# Major and minor: {<code>N, maj, min</code>};
# Seventh chords: {<code>N, maj, min, maj7, min7, 7</code>};
# Major and minor with inversions: {<code>N, maj, min, maj/3, min/b3, maj/5, min/5</code>}; or
# Seventh chords with inversions: {<code>N, maj, min, maj7, min7, 7, maj/3, min/b3, maj7/3, min7/b3, 7/3, maj/5, min/5, maj7/5, min7/5, 7/5, maj7/7, min7/b7, 7/b7</code>}.

With the exception of no-chords, calculating the vocabulary mapping involves examining the root note, the bass note, and the relative interval structure of the chord labels. A mapping exists if both the root notes and bass notes match, and the structure of the output label is the largest possible subset of the input label given the vocabulary. For instance, in the major and minor case, <code>G:7(#9)</code> is mapped to <code>G:maj</code> because the interval set of <code>G:maj</code>, {<code>1,3,5</code>}, is a subset of the interval set of the <code>G:7(#9)</code>, {<code>1,3,5,b7,#9</code>}. In the seventh-chord case, <code>G:7(#9)</code> is mapped to <code>G:7</code> instead because the interval set of <code>G:7</code> {<code>1, 3, 5, b7</code>} is also a subset of <code>G:7(#9)</code> but is larger than <code>G:maj</code>. If a chord cannot be represented by a certain class, e.g., mapping a <code>D:aug</code> or <code>F:sus4(9)</code> to {<code>maj, min</code>}, the chord is excluded from the evaluation if it occurs in the ground-truth, and it is considered a mismatch if it occurs in an estimated annotation.

{|
|+ Most frequent chord qualities in the McGill ''Billboard'' corpus.
! Quality
! Freq. (%)
! Cum. Freq (%)
|-
|maj
|52
|52
|-
|min
|13
|65
|-
|7
|10
|75
|-
|min7
|8
|83
|-
|maj7
|3
|86
|-
|5
|2
|88
|-
|1
|2
|90
|-
|maj(9)
|1
|91
|-
|maj6
|1
|92
|-
|sus4
|1
|93
|-
|sus7
|1
|94
|-
|sus9
|1
|94
|-
|7(#9)
|1
|95
|-
|min9
|1
|96
|}

Our recommendations are motivated by the frequencies of chord qualities in the ''Billboard'' corpus (see table above), which is a balanced sample of American popular music from the 1950s through the 1990s (J.A. Burgoyne, Wild, and Fujinaga 2011). Pure major and minor chords alone account for 65 percent of all chords encountered, whereas augmented and diminished triads account for 0.2 percent or less of the corpus each. Our arguments for our particular seventh-chord vocabulary as opposed to the set of all tetrads follows similar reasoning; our proposed vocabulary accounts for 86 percent of all chords, whereas no other standard type of seventh chord accounts for more than 0.2 percent of the corpus. In future years, the table suggests that we might consider introducing vocabularies including power chords, and possibly suspended chords or added sixths and ninths as well.

== Chord Segmentation ==

Besides CSR, the chord transcription literature includes several other metrics for evaluating chord transcriptions, which mainly focus on the segmentation of the automatic transcription. We propose to include the directional Hamming distance in the evaluation. The directional Hamming distance is calculated by finding for each annotated segment the maximally overlapping segment in the other annotation, and then summing the differences ((S. A. Abdallah et al. 2005); (Mauch 2010, §2.3.3)). Depending on the order of application, the directional Hamming distance yields a measure of over- or under segmentation. Both directions can be combined to yield an overall quality metric (Christopher Harte 2010, §8.3.2):

<math>Q = 1 - \frac{\text{maximum of directional Hamming distances in either direction}} {\text{total duration of song}}</math>

= Submission Format =

== Audio Format ==

Audio tracks in the training directory will be encoded as 44.1 kHz 16bit mono WAV files.

== I/O Format ==

The algorithms should output text files with a similar format to that used in the ground truth transcriptions. That is to say, they should be flat text files with chord segment labels and times arranged thus:

<pre>start_time end_time chord_label</pre>
with elements separated by white spaces, times given in seconds, chord labels corresponding to the syntax described by C. Harte et al. (2005), and one chord segment per line. As in all benchmarks after 2008, end times are a mandatory component of the output. For the evaluation process we will assume enharmonic equivalence for chord roots. We will no longer accept participants who would only like to be evaluated on major/minor chords and want to use the number format.

== Command line calling format ==

Submissions using machine learning models must also submit their trained models. Training on the evaluation server is no longer supported starting from this year. We will execute the following commands for testing:

<pre>
your_program prepare
your_program do_chord_identification "/path/to/input1.wav" "/path/to/output1.wav.txt"
your_program do_chord_identification "/path/to/input2.wav" "/path/to/output2.wav.txt"
...
</pre>

In the results directory, there should be one file for each testfile with same name as the test file + <code>.txt</code>. Programs can use the folder <code>/app/temp</code> if they need to keep temporary cache files or to download pretrained models. Standard output and standard error will be logged.

No internet access is allowed during the inference stage (<code>do_chord_identification</code>). Please contact us if your model requires internet access (e.g., model API call) during inference.

== Packaging submissions ==

* You can directly upload your submission (up to 5GB) to the MIREX submission site. If you need more storage you may contact the evaluation organizers.

=== Non-docker submission ===

* We recommend the participants submit a docker image to ensure that the evaluation team can easily run it.
* Non-docker submissions should contain a README file to include the setup procedure.

=== Docker Submission ===

* A docker submission will be deployed and evaluated automatically with <code>docker run</code>

Here are the docker commands that will be used to evaluate all systems:

<pre>
docker run -v temp_folder:/app/temp your_image_name prepare
docker run -v temp_folder:/app/temp -v dataset_folder:/app/data do_chord_identification "/app/data/input1.wav" "/app/data/output1.wav.txt"
docker run -v temp_folder:/app/temp -v dataset_folder:/app/data do_chord_identification "/app/data/input2.wav" "/app/data/output2.wav.txt"
...
</pre>

==== Example project ====

A sample project containing code and <code>Dockerfile</code> for a simple audio chord estimation baseline can be found in [https://github.com/futuremirex/audio_chord_estimation_sample_project https://github.com/futuremirex/audio_chord_estimation_sample_project].

Notice that this is not a compiled docker image - you need to build the image by your own before submitting it.

= Time and Hardware limits =

A Linux server with one Nvidia GeForce RTX 3090 is used for evaluation. CPU, OS, and memory specifications will be announced later.

Time limit: within 5 times the total duration of the test set.

= Bibliography =

Abdallah, Samer A., Katy Noland, Mark B. Sandler, Michael Casey, and Christophe Rhodes. 2005. “Theory and Evaluation of a Bayesian Music Structure Extractor.” In ''Proceedings of the International Society for Music Information Retrieval Conference'', 420–425.

Burgoyne, J. A., J. Wild, and I. Fujinaga. 2011. “An expert ground truth set for audio chord recognition and music analysis.” In ''Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR)'', 633–638.

Burgoyne, John Ashley. 2012. “Stochastic Processes and Database-Driven Musicology.” Ph.D. diss. Montréal, Québec, Canada: McGill University.

Haas, W. B. de, and John~Ashley Burgoyne. 2012. ''Parsing the Billboard Chord Transcriptions''. Technical report UU-CS- 2012-018, Department of Information and Computing Sciences, Utrecht University.

Harte, C., M. Sandler, S. Abdallah, and E. Gómez. 2005. “Symbolic representation of musical chords: A proposed syntax for text annotations.” In ''Proceedings of the 6th International Society for Music Information Retrieval Conference (ISMIR)'', 66–71.

Harte, Christopher. 2010. “Towards automatic extraction of harmony information from music signals.” Ph.D. diss. Queen Mary, University of London.

Mauch, Matthias. 2010. “Automatic Chord Transcription from Audio Using Computational Models of Musical Context.” Ph.D. diss. Queen Mary University of London.

Pauwels, Johan, and Geoffroy Peeters. 2013. “Evaluating automatically estimated chord sequences.” In ''Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)''. Vancouver, British Columbia, Canada.

2026:New Task Proposals

2026-06-16T20:12:31Z

Junyan:

These are the accepted task proposals for MIREX 2026.

==Task Descriptions==

Accepted challenge proposals
* [[2026:Audio-to-Score Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Audio Instrument Recognition]] <TC: [mailto:wenye.ma@mail.mcgill.ca Wenye Ma]>
* [[2026:Rhythm Game Chart Generation]] <TC: [mailto:ziyun.liu@univ-lille.fr Ziyun Liu] & [mailto:carolina.carusi@kaist.ac.kr Carolina Carusi]>
* [[2026:Music Evaluation via CMI-RewardBench]] <TC: [mailto:yinghao.ma@qmul.ac.uk Yinghao Ma], Yonghyun Kim, Haiwen Xia, Junwon Lee>
* [[2026:AI-Generated Music Detection]] <TC: [mailto:ldzhangyx@outlook.com Yixiao Zhang] & [mailto:you.zhang@rochester.edu Neil Zhang]>
* [[2026:Music Performance Difficulty Prediction]] <TC: [mailto:huan.zhang@qmul.ac.uk Huan Zhang] & [mailto:pedro@songscription.ai Pedro Ramoneda]>

Accepted task captain proposals
* [[2026:Lyrics Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>

Task captains may edit the names before Jul 1.

==Continued Tasks==

Modern MIR Tasks
* [[2026:Symbolic Music Generation]]

Standardized Evaluation Suites
* [[2026:Audio Chord Estimation]]
* [[2026:Audio Beat Tracking]]
* [[2026:Audio Key Detection]]
* [[2026:Audio Downbeat Tracking]]
* [[2026:Music Structure Analysis]]
* [[2026:Lyrics-to-Audio Alignment]]

User:Xinyue2896

2026-06-16T01:21:35Z

Junyan: Creating user page for new user.

Xinyue Li is a Music AI scientist.

User talk:Xinyue2896

2026-06-16T01:21:35Z

Junyan: Welcome!

2026:New Task Proposals

2026-06-09T21:10:15Z

Junyan: /* Task Descriptions */

2026:New Task Proposals

2026-06-05T17:32:15Z

Junyan: /* Task Descriptions */

These are the accepted task proposals for MIREX 2026.

==Task Descriptions==

Accepted challenge proposals
* [[2026:Audio-to-Score Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Audio Instrument Recognition]] <TC: [mailto:wenye.ma@mail.mcgill.ca Wenye Ma]>
* [[2026:Rhythm Game Chart Generation]] <TC: [mailto:ziyun.liu@univ-lille.fr Ziyun Liu] & [mailto:carolina.carusi@kaist.ac.kr Carolina Carusi]>
* [[2026:Music Evaluation via CMI-RewardBench and Music Arena]] <TC: [mailto:yinghao.ma@qmul.ac.uk Yinghao Ma], Yonghyun Kim, Haiwen Xia, Junwon Lee>
* [[2026:AI-Generated Music Detection]] <TC: [mailto:ldzhangyx@outlook.com Yixiao Zhang] & [mailto:you.zhang@rochester.edu Neil Zhang]>
* [[2026:Music Performance Difficulty Prediction]] <TC: [mailto:huan.zhang@qmul.ac.uk Huan Zhang] & [mailto:pedro@songscription.ai Pedro Ramoneda]>

Accepted task captain proposals
* [[2026:Lyrics Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>

Task captains may edit the names before Jul 1.

2026:New Task Proposals

2026-05-30T14:56:48Z

Junyan: /* Task Descriptions */

These are the accepted task proposals for MIREX 2026.

==Task Descriptions==

Accepted challenge proposals
* [[2026:Audio-to-Score Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Audio Instrument Recognition]] <TC: [mailto:wenye.ma@mail.mcgill.ca Wenye Ma]>
* [[2026:Rhythm Game Chart Generation]] <TC: [mailto:ziyun.liu@univ-lille.fr Ziyun Liu] & [mailto:carolina.carusi@kaist.ac.kr Carolina Carusi]>
* [[2026:Music Evaluation via CMI-RewardBench and Music Arena]] <TC: [mailto:yinghao.ma@qmul.ac.uk Yinghao Ma], Yonghyun Kim, Haiwen Xia, Junwon Lee, Chris Donahue>
* [[2026:AI-Generated Music Detection]] <TC: [mailto:ldzhangyx@outlook.com Yixiao Zhang] & [mailto:you.zhang@rochester.edu Neil Zhang]>
* [[2026:Music Performance Difficulty Prediction]] <TC: [mailto:huan.zhang@qmul.ac.uk Huan Zhang] & [mailto:pedro@songscription.ai Pedro Ramoneda]>

Accepted task captain proposals
* [[2026:Lyrics Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>

Task captains may edit the

User:Ziyunliu

2026-05-29T03:14:58Z

Junyan: Creating user page for new user.

I’m a PhD student in the Algomus and MINT teams at CRIStAL, Université de Lille, working on curiosity driven and lifelong learning for mixed initiative musical co-creativity in the scope of the ANR project MICCDroP. Previously, I did MS in Music and Technology and BS in Mathematical Sciences with minor in Physics at Carnegie Mellon University.

User talk:Ziyunliu

2026-05-29T03:14:58Z

Junyan: Welcome!

User:Alexandre DHooge

2026-05-29T03:11:07Z

Junyan: Creating user page for new user.

MIREX 2026 task captain account application request.

User talk:Alexandre DHooge

2026-05-29T03:11:07Z

Junyan: Welcome!

User:Wenye Ma

2026-05-29T03:10:58Z

Junyan: Creating user page for new user.

MIREX 2026 task captain account application request.

User talk:Wenye Ma

2026-05-29T03:10:58Z

Junyan: Welcome!

User:Carolina Carusi

2026-05-29T03:10:42Z

Junyan: Creating user page for new user.

MIREX 2026 task captain account application request.

User talk:Carolina Carusi

2026-05-29T03:10:42Z

Junyan: Welcome!

User:Yinghao Ma

2026-05-29T03:10:35Z

Junyan: Creating user page for new user.

MIREX 2026 task captain account application request.

User talk:Yinghao Ma

2026-05-29T03:10:35Z

Junyan: Welcome!

User:Yixiao Zhang

2026-05-29T03:10:27Z

Junyan: Creating user page for new user.

MIREX 2026 task captain account application request.

User talk:Yixiao Zhang

2026-05-29T03:10:27Z

Junyan: Welcome!

User:Huan Zhang

2026-05-29T03:10:18Z

Junyan: Creating user page for new user.

MIREX 2026 task captain account application request.

User talk:Huan Zhang

2026-05-29T03:10:18Z

Junyan: Welcome!

User:Pedro Ramoneda

2026-05-29T03:09:48Z

Junyan: Creating user page for new user.

MIREX 2026 task captain account application request.

User talk:Pedro Ramoneda

2026-05-29T03:09:48Z

Junyan: Welcome!

2026:New Task Proposals

2026-05-29T02:15:35Z

Junyan:

These are the accepted task proposals for MIREX 2026.

==Task Descriptions==

Accepted challenge proposals
* [[2026:Audio-to-Score Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Audio Instrument Recognition]] <TC: [mailto:wenye.ma@mail.mcgill.ca Wenye Ma]>
* [[2026:Rhythm Game Chart Generation]] <TC: [mailto:ziyun.liu@univ-lille.fr Ziyun Liu] & [mailto:carolina.carusi@kaist.ac.kr Carolina Carusi]>
* [[2026:Music Evaluation via CMI-RewardBench and Music Arena]] <TC: [mailto:yinghao.ma@qmul.ac.uk Yinghao Ma], Yonghyun Kim, Haiwen Xia, Junwon Lee, Chris Donahue>
* [[2026:AI-Generated Music Detection]] <TC: [mailto:ldzhangyx@outlook.com Yixiao Zhang]>
* [[2026:Music Performance Difficulty Prediction]] <TC: [mailto:huan.zhang@qmul.ac.uk Huan Zhang] & [mailto:pedro@songscription.ai Pedro Ramoneda]>

Accepted task captain proposals
* [[2026:Lyrics Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>

2026:New Task Proposals

2026-05-29T02:15:16Z

Junyan: /* Task Descriptions */

These are the accepted task proposals for MIREX 2026.
These are the accepted task proposals for MIREX 2026.

==Task Descriptions==

Accepted challenge proposals
* [[2026:Audio-to-Score Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Audio Instrument Recognition]] <TC: [mailto:wenye.ma@mail.mcgill.ca Wenye Ma]>
* [[2026:Rhythm Game Chart Generation]] <TC: [mailto:ziyun.liu@univ-lille.fr Ziyun Liu] & [mailto:carolina.carusi@kaist.ac.kr Carolina Carusi]>
* [[2026:Music Evaluation via CMI-RewardBench and Music Arena]] <TC: [mailto:yinghao.ma@qmul.ac.uk Yinghao Ma], Yonghyun Kim, Haiwen Xia, Junwon Lee, Chris Donahue>
* [[2026:AI-Generated Music Detection]] <TC: [mailto:ldzhangyx@outlook.com Yixiao Zhang]>
* [[2026:Music Performance Difficulty Prediction]] <TC: [mailto:huan.zhang@qmul.ac.uk Huan Zhang] & [mailto:pedro@songscription.ai Pedro Ramoneda]>

Accepted task captain proposals
* [[2026:Lyrics Transcription]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>

2026:New Task Proposals

2026-05-29T02:13:20Z

Junyan: Created page with "These are the accepted task proposals for MIREX 2026. ==Task Descriptions== Accepted challenge proposals * 2026:Audio-to-Score Transcription (A2S) <TC: [mailto:dhooge@gbu...."

These are the accepted task proposals for MIREX 2026.
==Task Descriptions==
Accepted challenge proposals
* [[2026:Audio-to-Score Transcription (A2S)]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>
* [[2026:Audio Instrument Recognition]] <TC: [mailto:wenye.ma@mail.mcgill.ca Wenye Ma]>
* [[2026:Rhythm Game Chart Generation]] <TC: [mailto:ziyun.liu@univ-lille.fr Ziyun Liu] & [mailto:carolina.carusi@kaist.ac.kr Carolina Carusi]>
* [[2026:Music Evaluation via CMI-RewardBench and Music Arena]] <TC: [mailto:yinghao.ma@qmul.ac.uk Yinghao Ma], Yonghyun Kim, Haiwen Xia, Junwon Lee, Chris Donahue>
* [[2026:AI-Generated Music Detection]] <TC: [mailto:ldzhangyx@outlook.com Yixiao Zhang]>
* [[2026:Music Performance Difficulty Prediction]] <TC: [mailto:huan.zhang@qmul.ac.uk Huan Zhang] & [mailto:pedro@songscription.ai Pedro Ramoneda]>
Accepted task captain proposals
* [[2026:Automatic Lyrics Transcription (ALT)]] <TC: [mailto:dhooge@gbu.edu.cn Alexandre D'Hooge]>

MIREX HOME

2026-03-16T06:00:46Z

Junyan: /* Welcome to MIREX 2026 */

==Welcome to MIREX 2026==

MIREX (Music Information Retrieval Evaluation eXchange) is an annual community evaluation campaign held in conjunction with the [https://ismir.net/ International Society for Music Information Retrieval Conference (ISMIR)]. This year, the conference will be held in [https://ismir2026.ismir.net/ Abu Dhabi, UAE] from November 8–12, 2026, and may include an online component.

In a long run, we want to make MIREX a platform for researchers to share their latest research results, to compare their systems with others, and to promote the development of the field.

==Call for Challenges==

We invite the ISMIR community to propose new research challenges that address cutting-edge problems in Music Information Retrieval (MIR). These challenges should aim to push the boundaries of current research and foster innovation in the field. We also welcome challenge sponsors from both industry and research institutions, particularly those willing to contribute datasets and computational resources to support the competition.

For the format and requirements for the challenge proposal, please go to [[2026:Call for Challenges]].

Task Captain Responsibilities:

* Register on the [https://www.music-ir.org/mirex MIREX Wiki] and maintain a task description page.
* Collect submissions via the MIREX submission server (or provide customized submission instructions).
* Execute and evaluate the submissions.
* Report results to MIREX and create a results page on the MIREX Wiki.
* (Optional) Present a MIREX task captain poster at the Late-Breaking and Demo (LBD) session at ISMIR 2026.

===What's new:===

In MIREX 2026, we introduce two special calls for challenges:

1. Novel evaluation pipelines

There is a growing need for new evaluation methodologies. We welcome challenge proposals that introduce novel evaluation pipelines, including but not limited to the evaluation of generative models, interactive systems, and other emerging paradigms.

2. Evaluation under limited resources

In addition to state-of-the-art systems, MIREX aims to better support researchers and research areas with limited resources. We welcome challenge proposals that focus on underrepresented research areas, music genres, or research communities.

Furthermore, all task captains are encouraged to report key resource indicators for each submission, including:

* training data size
* model size
* other computational resources used in model development

==How to Participate==

See also the general [[Submission Guidelines]].

* Read the [[Participant Agreement]] and task description carefully.
* Program your system.
* Write a 2-4 page extended abstract PDF describing your system.
* Submit your system and extended abstract to the [http://futuremirex.com/submission MIREX submission site].
* Top-performing teams will have the opportunity to present their MIREX posters at the LBD session at ISMIR 2026.

==Important Dates==

* Challenge proposals due: May 15, 2026
* Notification of acceptance: May 29, 2026
* Submission open: Jul 1, 2026
* Submission close: Oct 1, 2026 (Some tasks may have a different deadline; see task descriptions)
* Result published: Oct 15, 2026 (Some tasks may have a different deadline; see task descriptions)

==Contact Us==

====Email====

For general questions, feedback, and suggestions, please send messages to our mailing list [mailto:future-mirex@googlegroups.com future-mirex@googlegroups.com].

For task-specific questions, we have listed the email for each task captain [[MIREX_HOME#Task_Descriptions|here]].

====Discord Server====

For real-time discussion with the MIREX organizers or task captains, you may join our [https://discord.gg/vC2YWX29sC discord server].

Notice: some task captains are not in the discord server.

====Repositories====

Open-source evaluation pipelines: https://github.com/ismir-mirex/mirex-evaluation

Github organization: https://github.com/ismir-mirex/

====LinkedIn Organization Page====

You may visit our LinkedIn organization page [https://www.linkedin.com/company/future-mirex/ here].

We are looking forward to seeing you at MIREX 2026!

Future MIREX Team, 2026

MIREX 2026 Organizers:
* Junyan Jiang, New York University
* Gus Xia, MBZUAI
* Akira Maezawa, Yamaha
* Ziyu Wang, New York University
* Yixiao Zhang, ByteDance Inc.
* Ruibin Yuan, Hong Kong University of Science and Technology
* J. Stephen Downie, University of Illinois

2026:Call for Challenges

2026-03-16T05:35:49Z

Junyan: /* Deadlines */

MIREX HOME

2026-03-16T05:35:27Z

Junyan: /* Important Dates */

==Welcome to MIREX 2026==

After a break of 3 years, we want to bring back the MIREX (Music Information Retrieval Evaluation eXchange) competition starting from 2024. We want to bring in new tasks, benchmarks, and datasets in response to the rapid development of computer music research.

The MIREX community will hold its annual meeting as part of [https://ismir.net/ The International Society for Music Information Retrieval Conference]. This year, the conference will be held in [https://ismir2026.ismir.net/ Abu Dhabi, UAE] from November 8-12, 2026, possibly with an online component.

In a long run, we want to make MIREX a platform for researchers to share their latest research results, to compare their systems with others, and to promote the development of the field.

==Call for Challenges==

We invite the ISMIR community to propose new research challenges that address cutting-edge problems in Music Information Retrieval (MIR). These challenges should aim to push the boundaries of current research and foster innovation in the field. We also welcome challenge sponsors from both industry and research institutions, particularly those willing to contribute datasets and computational resources to support the competition.

For the format and requirements for the challenge proposal, please go to [[2026:Call for Challenges]].

Task Captain Responsibilities:

* Register on the [https://www.music-ir.org/mirex MIREX Wiki] and maintain a task description page.
* Collect submissions via the MIREX submission server (or provide customized submission instructions).
* Execute and evaluate the submissions.
* Report results to MIREX and create a results page on the MIREX Wiki.
* (Optional) Present a MIREX task captain poster at the Late-Breaking and Demo (LBD) session at ISMIR 2026.

===What's new:===

In MIREX 2026, we introduce two special calls for challenges:

1. Novel evaluation pipelines

There is a growing need for new evaluation methodologies. We welcome challenge proposals that introduce novel evaluation pipelines, including but not limited to the evaluation of generative models, interactive systems, and other emerging paradigms.

2. Evaluation under limited resources

In addition to state-of-the-art systems, MIREX aims to better support researchers and research areas with limited resources. We welcome challenge proposals that focus on underrepresented research areas, music genres, or research communities.

Furthermore, all task captains are encouraged to report key resource indicators for each submission, including:

* training data size
* model size
* other computational resources used in model development

==How to Participate==

See also the general [[Submission Guidelines]].

* Read the [[Participant Agreement]] and task description carefully.
* Program your system.
* Write a 2-4 page extended abstract PDF describing your system.
* Submit your system and extended abstract to the [http://futuremirex.com/submission MIREX submission site].
* Top-performing teams will have the opportunity to present their MIREX posters at the LBD session at ISMIR 2026.

==Important Dates==

* Challenge proposals due: May 15, 2026
* Notification of acceptance: May 29, 2026
* Submission open: Jul 1, 2026
* Submission close: Oct 1, 2026 (Some tasks may have a different deadline; see task descriptions)
* Result published: Oct 15, 2026 (Some tasks may have a different deadline; see task descriptions)

==Contact Us==

====Email====

For general questions, feedback, and suggestions, please send messages to our mailing list [mailto:future-mirex@googlegroups.com future-mirex@googlegroups.com].

For task-specific questions, we have listed the email for each task captain [[MIREX_HOME#Task_Descriptions|here]].

====Discord Server====

For real-time discussion with the MIREX organizers or task captains, you may join our [https://discord.gg/vC2YWX29sC discord server].

Notice: some task captains are not in the discord server.

====Repositories====

Open-source evaluation pipelines: https://github.com/ismir-mirex/mirex-evaluation

Github organization: https://github.com/ismir-mirex/

====LinkedIn Organization Page====

You may visit our LinkedIn organization page [https://www.linkedin.com/company/future-mirex/ here].

We are looking forward to seeing you at MIREX 2026!

Future MIREX Team, 2026

MIREX 2026 Organizers:
* Junyan Jiang, New York University
* Gus Xia, MBZUAI
* Akira Maezawa, Yamaha
* Ziyu Wang, New York University
* Yixiao Zhang, ByteDance Inc.
* Ruibin Yuan, Hong Kong University of Science and Technology
* J. Stephen Downie, University of Illinois

2026:Call for Challenges

2026-03-16T04:53:21Z

Junyan:

2026:Call for Challenges

2026-03-16T04:48:46Z

Junyan: /* Call for Challenge Proposals & Task Captains */

2025:Previous Tasks

2026-03-16T04:47:36Z

Junyan: /* List of Previous Tasks */

==List of Previous Tasks==

This is a list of previous MIREX tasks for reference only.

* [[2025:RenCon]] (Expressive Piano Performance Rendering Competition) <TC: [mailto:huan.zhang@qmul.ac.uk Huan Zhang]>
* [[2025:Music Reasoning QA]] <TC: [mailto:yinghao.ma@qmul.ac.uk Yinghao Ma]>
* [[2024:Symbolic Music Generation]] <TC: [mailto:ziyu.wang@nyu.edu Ziyu Wang]>
* [[2024:Music Audio Generation]] <TC: [mailto:ruibiny@alumni.cmu.edu Ruibin Yuan]>
* [[2024:Music Description & Captioning]] <TC: [mailto:yixiao.zhang@qmul.ac.uk Yixiao Zhang]>
* [[2024:Polyphonic Transcription]] <TC: [mailto:yujia.yan@rochester.edu Yujia Yan] & [mailto:ziyu.wang@nyu.edu Ziyu Wang]>
* [[2024:Singing Voice Deepfake Detection]] <TC: [mailto:you.zhang@rochester.edu Neil Zhang] & [mailto:yixiao.zhang@qmul.ac.uk Yixiao Zhang]>
* [[2024:Cover Song Identification]] <TC: [mailto:x.du@rochester.edu Xingjian Du] & [mailto:ruibiny@alumni.cmu.edu Ruibin Yuan]>
* [[2024:Lyrics-to-Audio Alignment]] <TC: [mailto:jj2731@nyu.edu Junyan Jiang]>
* [[2021:Audio Beat Tracking]]
* [[2021:Audio Chord Estimation]]
* [[2021:Audio Cover Song Identification]]
* [[2021:Audio Downbeat Estimation]]
* [[2021:Audio Fingerprinting]]
* [[2021:Audio Key Detection]]
* [[2021:Audio Melody Extraction]]
* [[2021:Audio Onset Detection]]
* [[2021:Audio Tag Classification]]
* [[2021:Audio Tempo Estimation]]
* [[2021:Automatic Lyrics Transcription]]
* [[2021:Drum Transcription]]
* [[2021:Multiple Fundamental Frequency Estimation & Tracking]]
* [[2021:Music Detection]]
* [[2021:Patterns for Prediction]] (offshoot of [[2017:Discovery of Repeated Themes & Sections]])
* [[2021:Query by Singing/Humming]]
* [[2021:Query by Tapping]]
* [[2021:Real-time Audio to Score Alignment (a.k.a Score Following)]]
* [[2021:Set List Identification]]
* [[2021:Structural Segmentation]]
* [[2020:Singing_Transcription_from_Polyphonic_Music]]
* [[2018:Music and/or Speech Detection]]
* [[2016:GC16UX|2016:Grand Challenge on User Experience]]
* [[2016:Audio Offset Detection]]

2026:Call for Challenges

2026-03-16T04:45:52Z

Junyan: /* Call for Challenge Proposals & Task Captains */

==Call for Challenge Proposals & Task Captains==

We invite the ISMIR community to participate in shaping the future of Music Information Retrieval (MIR) by either '''proposing new research challenges''' or '''volunteering as task captains for [https://www.music-ir.org/mirex/wiki/2025:Previous_Tasks existing ones]'''.

* New challenge proposals should aim to address cutting-edge problems and push the boundaries of current MIR research.
* Task captains for established tasks are encouraged to help revitalize them—potentially by updating evaluation methodologies, datasets, or other aspects to reflect recent advances in the field.

We also welcome challenge sponsors from both industry and academia, particularly those willing to contribute datasets, evaluation tools, or computational resources to support the competition. This may include proprietary test sets, i.e., datasets that remain private and are not disclosed to participants or MIREX organizers.

What's new:

In MIREX 2026, we introduce two special calls for challenges:

1. Novel evaluation pipelines

There is a growing need for new evaluation methodologies. We welcome challenge proposals that introduce novel evaluation pipelines, including but not limited to the evaluation of generative models, interactive systems, and other emerging paradigms.

2. Evaluation under limited resources

In addition to state-of-the-art systems, MIREX aims to better support researchers and research areas with limited resources. We welcome challenge proposals that focus on underrepresented research areas, music genres, or research communities.

For all tasks, task submissions are encouraged to report key resource indicators, including:

training data size
model size
other computational resources used in model development
Task captains are encouraged to summarize and report this information in the task results page.

Rules
For both new and established tasks, all proposals must have an assigned task captain responsible for evaluating participant submissions.

Task Captain Responsibilities:

Register on the MIREX Wiki and maintain the task description page.
Collect submissions via the MIREX submission server (or provide customized submission instructions).
Execute and evaluate the submissions.
Report results to MIREX and create a results page on the MIREX Wiki.
(Optional) Present a MIREX task captain poster at the Late-Breaking and Demo (LBD) session at ISMIR 2026.
Submission Format
A call for challenge proposal must contain the following content:

Title of the new task
Abstract
Task description
Significance of the task
Evaluation criteria
Datasets and resources provided
Requirements for submission
Task captain information (name, title, affiliation, email, and MIREX Wiki username)
Long-term plan (your willingness and availability to maintain the task in coming years)

A task captain proposal should contain the following content:

Title of the existing task
Abstract
Task description
Evaluation criteria, if different from previous MIREXes
Datasets and resources provided, if different from previous MIREXes
Requirements for submission, if different from previous MIREXes
Task captain information (name, title, affiliation, email, and MIREX Wiki username)
Long-term plan (your willingness and availability to maintain the task in coming years)
Please submit a 1-4 page PDF to the MIREX organizing committee future-mirex@googlegroups.com before the due date.

Multiple submissions
A single individual may serve as the task captain for multiple tasks.
Conversely, a single task may be co-managed by multiple task captains.
If we receive similar proposals from different authors, we may reach out to coordinate a joint effort to co-host the task.
Deadlines
Challenge proposals due: May 22, 2026
Notification of acceptance: May 29, 2026
Contact Us
We look forward to receiving your innovative and impactful challenge proposals. If you have any questions, please do not hesitate to contact the MIREX organizing committee future-mirex@googlegroups.com.

MIREX 2026 Organizers:

Gus Xia, MBZUAI
Junyan Jiang, New York University
Akira Maezawa, Yamaha
Ziyu Wang, New York University
Yixiao Zhang, ByteDance
Ruibin Yuan, Hong Kong University of Science and Technology
J. Stephen Downie, University of Illinois
We also welcome challenge ''sponsors'' from both industry and academia, particularly those willing to contribute datasets, evaluation tools, or computational resources to support the competition. This may include proprietary test sets, i.e., datasets that remain private and are not disclosed to participants or MIREX organizers.

'''What's new:'''

In MIREX 2026, we introduce two special calls for challenges:

1. Novel evaluation pipelines

There is a growing need for new evaluation methodologies. We welcome challenge proposals that introduce novel evaluation pipelines, including but not limited to the evaluation of generative models, interactive systems, and other emerging paradigms.

2. Evaluation under limited resources

In addition to state-of-the-art systems, MIREX aims to better support researchers and research areas with limited resources. We welcome challenge proposals that focus on underrepresented research areas, music genres, or research communities.

For all tasks, task submissions are encouraged to report key resource indicators, including:

* training data size
* model size
* other computational resources used in model development

Task captains are encouraged to summarize and report this information in the task results page.

==Rules==

For both new and established tasks, all proposals must have an assigned task captain responsible for evaluating participant submissions.

'''Task Captain Responsibilities:'''
* Register on the [https://www.music-ir.org/mirex MIREX Wiki] and maintain the task description page.
* Collect submissions via the MIREX submission server (or provide customized submission instructions).
* Execute and evaluate the submissions.
* Report results to MIREX and create a results page on the MIREX Wiki.
* (Optional) Present a MIREX task captain poster at the Late-Breaking and Demo (LBD) session at ISMIR 2026.

==Submission Format==

A call for challenge proposal must contain the following content:

# Title of the new task
# Abstract
# Task description
# Significance of the task
# Evaluation criteria
# Datasets and resources provided
# Requirements for submission
# Task captain information (name, title, affiliation, email, and MIREX Wiki username)
# Long-term plan (your willingness and availability to maintain the task in coming years)

A task captain proposal should contain the following content:

# Title of the existing task
# Abstract
# Task description
# Evaluation criteria, if different from previous MIREXes
# Datasets and resources provided, if different from previous MIREXes
# Requirements for submission, if different from previous MIREXes
# Task captain information (name, title, affiliation, email, and MIREX Wiki username)
# Long-term plan (your willingness and availability to maintain the task in coming years)

Please submit a 1-4 page PDF to the MIREX organizing committee [mailto:future-mirex@googlegroups.com future-mirex@googlegroups.com] before the due date.

===Multiple submissions===
* A single individual may serve as the task captain for multiple tasks.
* Conversely, a single task may be co-managed by multiple task captains.
* If we receive similar proposals from different authors, we may reach out to coordinate a joint effort to co-host the task.

==Deadlines==

* Challenge proposals due: May 22, 2026
* Notification of acceptance: May 29, 2026

==Contact Us==

We look forward to receiving your innovative and impactful challenge proposals. If you have any questions, please do not hesitate to contact the MIREX organizing committee [mailto:future-mirex@googlegroups.com future-mirex@googlegroups.com].

MIREX 2026 Organizers:
* Gus Xia, MBZUAI
* Junyan Jiang, New York University
* Akira Maezawa, Yamaha
* Ziyu Wang, New York University
* Yixiao Zhang, ByteDance
* Ruibin Yuan, Hong Kong University of Science and Technology
* J. Stephen Downie, University of Illinois

2026:Call for Challenges

2026-03-16T04:43:40Z

Junyan:

==Call for Challenge Proposals & Task Captains==

We invite the ISMIR community to propose new research challenges that address cutting-edge problems in Music Information Retrieval (MIR). These challenges should aim to push the boundaries of current research and foster innovation in the field.

We also welcome challenge ''sponsors'' from both industry and academia, particularly those willing to contribute datasets, evaluation tools, or computational resources to support the competition. This may include proprietary test sets, i.e., datasets that remain private and are not disclosed to participants or MIREX organizers.

'''What's new:'''

In MIREX 2026, we introduce two special calls for challenges:

1. Novel evaluation pipelines

There is a growing need for new evaluation methodologies. We welcome challenge proposals that introduce novel evaluation pipelines, including but not limited to the evaluation of generative models, interactive systems, and other emerging paradigms.

2. Evaluation under limited resources

In addition to state-of-the-art systems, MIREX aims to better support researchers and research areas with limited resources. We welcome challenge proposals that focus on underrepresented research areas, music genres, or research communities.

For all tasks, task submissions are encouraged to report key resource indicators, including:

* training data size
* model size
* other computational resources used in model development

Task captains are encouraged to summarize and report this information in the task results page.

==Rules==

For both new and established tasks, all proposals must have an assigned task captain responsible for evaluating participant submissions.

'''Task Captain Responsibilities:'''
* Register on the [https://www.music-ir.org/mirex MIREX Wiki] and maintain the task description page.
* Collect submissions via the MIREX submission server (or provide customized submission instructions).
* Execute and evaluate the submissions.
* Report results to MIREX and create a results page on the MIREX Wiki.
* (Optional) Present a MIREX task captain poster at the Late-Breaking and Demo (LBD) session at ISMIR 2026.

==Submission Format==

A call for challenge proposal must contain the following content:

# Title of the new task
# Abstract
# Task description
# Significance of the task
# Evaluation criteria
# Datasets and resources provided
# Requirements for submission
# Task captain information (name, title, affiliation, email, and MIREX Wiki username)
# Long-term plan (your willingness and availability to maintain the task in coming years)

A task captain proposal should contain the following content:

# Title of the existing task
# Abstract
# Task description
# Evaluation criteria, if different from previous MIREXes
# Datasets and resources provided, if different from previous MIREXes
# Requirements for submission, if different from previous MIREXes
# Task captain information (name, title, affiliation, email, and MIREX Wiki username)
# Long-term plan (your willingness and availability to maintain the task in coming years)

Please submit a 1-4 page PDF to the MIREX organizing committee [mailto:future-mirex@googlegroups.com future-mirex@googlegroups.com] before the due date.

===Multiple submissions===
* A single individual may serve as the task captain for multiple tasks.
* Conversely, a single task may be co-managed by multiple task captains.
* If we receive similar proposals from different authors, we may reach out to coordinate a joint effort to co-host the task.

==Deadlines==

* Challenge proposals due: May 22, 2026
* Notification of acceptance: May 29, 2026

==Contact Us==

We look forward to receiving your innovative and impactful challenge proposals. If you have any questions, please do not hesitate to contact the MIREX organizing committee [mailto:future-mirex@googlegroups.com future-mirex@googlegroups.com].

MIREX 2026 Organizers:
* Gus Xia, MBZUAI
* Junyan Jiang, New York University
* Akira Maezawa, Yamaha
* Ziyu Wang, New York University
* Yixiao Zhang, ByteDance
* Ruibin Yuan, Hong Kong University of Science and Technology
* J. Stephen Downie, University of Illinois