2024:Symbolic Music Generation

Description

Symbolic music generation is a broad topic. It covers a wide range of tasks, including generation, harmonization, arrangement, instrumentation, and more. We have multiple ways to represent music data, and the evaluation metrics also vary. To define a MIREX challenge within this topic, we need to narrow our focus to specific subtasks that are both relevant to the community and feasible to evaluate effectively.

This year, we select the task to be piano accompaniment arrangement from a lead sheet. The lead sheet provides information about the melody, chord progression, and optional phrase labels. The goal is to generate a piano accompaniment that complements the lead melody. The music data consists of 8-measure segments in 4/4 meter, quantized to a sixteenth-note resolution. A more detailed description of the data structure is provided in the data format section. The genre of the lead sheets is broadly within western pop music (refer to the music examples for more detail).

Evaluation

We will evaluate the submitted algorithms through an online subjective double-blind test.

• We use a "potluck" test set. Before submitting the algorithm, each team is required to submit two lead sheets. The organizer team will supplement the lead sheet if necessary.
• Once the organizer team receives the lead sheet submission, a schedule for the subjective test will be made. Depending on the number of submissions, there could be more than one round of subjective test.
• We will use objective measurements only as a reference. The correlation between subjective and objective scores will be presented in the end.

Subjective Metrics

We will ask the subjects to listen to the original lead sheet first and then listen to the anonymized generations. Subjects will be asked to rate the generation according to the following criteria:

• Harmony correctness (5-point scale) (To be done)
• Creativity (5-point scale) (To be done)
• Naturalness (5-point scale) (To be done)
• Coherence and stylistic appropriateness? To be done.

Objective Metrics

We will report these metrics as reference:

• To be done.