Difference between revisions of "2025:Call for Challenges"
From MIREX Wiki
(→Call for Challenge Proposals & Task Captains) |
(→Call for Challenge Proposals & Task Captains) |
||
| (13 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
==Call for Challenge Proposals & Task Captains== | ==Call for Challenge Proposals & Task Captains== | ||
| − | + | Starting with MIREX 2025, we invite the ISMIR community to participate in shaping the future of Music Information Retrieval (MIR) by either '''proposing new research challenges''' or '''volunteering as task captains for [https://www.music-ir.org/mirex/wiki/2025:Previous_Tasks existing ones]'''. | |
| − | |||
| − | Starting with MIREX 2025, we invite the ISMIR community to participate in shaping the future of Music Information Retrieval (MIR) by either '''proposing new research challenges''' or '''volunteering as task captains for existing ones'''. | ||
* New challenge proposals should aim to address cutting-edge problems and push the boundaries of current MIR research. | * New challenge proposals should aim to address cutting-edge problems and push the boundaries of current MIR research. | ||
* Task captains for established tasks are encouraged to help revitalize them—potentially by updating evaluation methodologies, datasets, or other aspects to reflect recent advances in the field. | * Task captains for established tasks are encouraged to help revitalize them—potentially by updating evaluation methodologies, datasets, or other aspects to reflect recent advances in the field. | ||
| − | We also welcome challenge sponsors from both industry and academia, particularly those willing to contribute datasets, evaluation tools, or computational resources to support the competition. | + | We also welcome challenge ''sponsors'' from both industry and academia, particularly those willing to contribute datasets, evaluation tools, or computational resources to support the competition. This includes proprietary test sets—that is, datasets which will remain private and not be disclosed to either participants or MIREX organizers. |
==Rules for Year 2025== | ==Rules for Year 2025== | ||
| − | + | For both new and established tasks, all proposals must have an assigned task captain responsible for evaluating participant submissions. | |
| − | + | ||
| + | '''Task Captain Responsibilities:''' | ||
* Register on the [https://www.music-ir.org/mirex MIREX Wiki] and maintain the task description page. | * Register on the [https://www.music-ir.org/mirex MIREX Wiki] and maintain the task description page. | ||
* Collect submissions via the MIREX submission server (or provide customized submission instructions). | * Collect submissions via the MIREX submission server (or provide customized submission instructions). | ||
* Execute and evaluate the submissions. | * Execute and evaluate the submissions. | ||
* Report results to MIREX and create a results page on the MIREX Wiki. | * Report results to MIREX and create a results page on the MIREX Wiki. | ||
| − | (Optional) Present a MIREX task captain poster at the Late-Breaking and Demo (LBD) session at ISMIR 2025. | + | * (Optional) Present a MIREX task captain poster at the Late-Breaking and Demo (LBD) session at ISMIR 2025. |
| − | === | + | ===Subtasks=== |
| − | + | To invite research groups without access to large datasets or extensive computational resources, you are encouraged to create multiple subtasks that impose different restrictions on model size and dataset usage. Example: | |
Subtask 1: Model size < 200M. Training set: MusDB18 only. | Subtask 1: Model size < 200M. Training set: MusDB18 only. | ||
Subtask 2: Model size: any. Training set: any. | Subtask 2: Model size: any. Training set: any. | ||
| − | * | + | |
| + | ===Evaluation Methods=== | ||
| + | |||
| + | The task captain may choose freely how to evaluate the submissions. This includes but is not restricted to: | ||
| + | * Traditional MIREX evaluation pipelines | ||
| + | * Subjective evaluation (e.g., [[2024:Symbolic Music Generation]]) | ||
| + | * Live leaderboards (please refer to [[2025:Live Leaderboards]] for details) | ||
==Submission Format== | ==Submission Format== | ||
| Line 39: | Line 44: | ||
# Requirements for submission | # Requirements for submission | ||
# Task captain information (name, title, affiliation, email, and MIREX Wiki username) | # Task captain information (name, title, affiliation, email, and MIREX Wiki username) | ||
| + | # Long-term plan (your willingness and availability to maintain the task in coming years) | ||
| + | |||
A task captain proposal should contain the following content: | A task captain proposal should contain the following content: | ||
| Line 49: | Line 56: | ||
# Requirements for submission, if different from previous MIREXes | # Requirements for submission, if different from previous MIREXes | ||
# Task captain information (name, title, affiliation, email, and MIREX Wiki username) | # Task captain information (name, title, affiliation, email, and MIREX Wiki username) | ||
| + | # Long-term plan (your willingness and availability to maintain the task in coming years) | ||
| + | Please submit a 1-4 page PDF to the MIREX organizing committee [mailto:future-mirex@googlegroups.com future-mirex@googlegroups.com] before the due date. | ||
| − | + | ===Multiple submissions=== | |
| + | * A single individual may serve as the task captain for multiple tasks. | ||
| + | * Conversely, a single task may be co-managed by multiple task captains. | ||
| + | * If we receive similar proposals from different authors, we may reach out to coordinate a joint effort to co-host the task. | ||
==Deadlines== | ==Deadlines== | ||
| − | * Challenge proposals due: | + | * Challenge proposals due: May 15, 2025 |
| − | * Notification of acceptance: | + | * Notification of acceptance: May 25, 2025 |
==Contact Us== | ==Contact Us== | ||
| Line 63: | Line 75: | ||
MIREX 2025 Organizers: | MIREX 2025 Organizers: | ||
| + | * Gus Xia, MBZUAI | ||
* Junyan Jiang, New York University | * Junyan Jiang, New York University | ||
* Akira Maezawa, Yamaha | * Akira Maezawa, Yamaha | ||
| Line 69: | Line 82: | ||
* Ruibin Yuan, Hong Kong University of Science and Technology | * Ruibin Yuan, Hong Kong University of Science and Technology | ||
* J. Stephen Downie, University of Illinois | * J. Stephen Downie, University of Illinois | ||
| − | |||
Latest revision as of 07:54, 9 April 2025
Contents
Call for Challenge Proposals & Task Captains
Starting with MIREX 2025, we invite the ISMIR community to participate in shaping the future of Music Information Retrieval (MIR) by either proposing new research challenges or volunteering as task captains for existing ones.
- New challenge proposals should aim to address cutting-edge problems and push the boundaries of current MIR research.
- Task captains for established tasks are encouraged to help revitalize them—potentially by updating evaluation methodologies, datasets, or other aspects to reflect recent advances in the field.
We also welcome challenge sponsors from both industry and academia, particularly those willing to contribute datasets, evaluation tools, or computational resources to support the competition. This includes proprietary test sets—that is, datasets which will remain private and not be disclosed to either participants or MIREX organizers.
Rules for Year 2025
For both new and established tasks, all proposals must have an assigned task captain responsible for evaluating participant submissions.
Task Captain Responsibilities:
- Register on the MIREX Wiki and maintain the task description page.
- Collect submissions via the MIREX submission server (or provide customized submission instructions).
- Execute and evaluate the submissions.
- Report results to MIREX and create a results page on the MIREX Wiki.
- (Optional) Present a MIREX task captain poster at the Late-Breaking and Demo (LBD) session at ISMIR 2025.
Subtasks
To invite research groups without access to large datasets or extensive computational resources, you are encouraged to create multiple subtasks that impose different restrictions on model size and dataset usage. Example:
Subtask 1: Model size < 200M. Training set: MusDB18 only. Subtask 2: Model size: any. Training set: any.
Evaluation Methods
The task captain may choose freely how to evaluate the submissions. This includes but is not restricted to:
- Traditional MIREX evaluation pipelines
- Subjective evaluation (e.g., 2024:Symbolic Music Generation)
- Live leaderboards (please refer to 2025:Live Leaderboards for details)
Submission Format
A call for challenge proposal must contain the following content:
- Title of the new task
- Abstract
- Task description
- Significance of the task
- Evaluation criteria
- Datasets and resources provided
- Requirements for submission
- Task captain information (name, title, affiliation, email, and MIREX Wiki username)
- Long-term plan (your willingness and availability to maintain the task in coming years)
A task captain proposal should contain the following content:
- Title of the existing task
- Abstract
- Task description
- Evaluation criteria, if different from previous MIREXes
- Datasets and resources provided, if different from previous MIREXes
- Requirements for submission, if different from previous MIREXes
- Task captain information (name, title, affiliation, email, and MIREX Wiki username)
- Long-term plan (your willingness and availability to maintain the task in coming years)
Please submit a 1-4 page PDF to the MIREX organizing committee future-mirex@googlegroups.com before the due date.
Multiple submissions
- A single individual may serve as the task captain for multiple tasks.
- Conversely, a single task may be co-managed by multiple task captains.
- If we receive similar proposals from different authors, we may reach out to coordinate a joint effort to co-host the task.
Deadlines
- Challenge proposals due: May 15, 2025
- Notification of acceptance: May 25, 2025
Contact Us
We look forward to receiving your innovative and impactful challenge proposals. If you have any questions, please do not hesitate to contact the MIREX organizing committee future-mirex@googlegroups.com.
MIREX 2025 Organizers:
- Gus Xia, MBZUAI
- Junyan Jiang, New York University
- Akira Maezawa, Yamaha
- Ziyu Wang, New York University
- Yixiao Zhang, ByteDance
- Ruibin Yuan, Hong Kong University of Science and Technology
- J. Stephen Downie, University of Illinois
