Difference between revisions of "2024:Symbolic Music Generation"
Zizzi wang (talk | contribs) (→Subjective Metrics) |
Zizzi wang (talk | contribs) (→Subjective Metrics) |
||
| Line 15: | Line 15: | ||
===Subjective Metrics=== | ===Subjective Metrics=== | ||
We will ask the subjects to listen to the original lead sheet first and then listen to the anonymized generations. Subjects will be asked to rate the generation according to the following criteria: | We will ask the subjects to listen to the original lead sheet first and then listen to the anonymized generations. Subjects will be asked to rate the generation according to the following criteria: | ||
| − | * Harmony correctness | + | * Harmony correctness (5-point scale) (To be done) |
| − | * Creativity | + | * Creativity (5-point scale) (To be done) |
| − | * Naturalness | + | * Naturalness (5-point scale) (To be done) |
| − | * To be done | + | * To be done |
===Objective Metrics=== | ===Objective Metrics=== | ||
Revision as of 06:15, 25 August 2024
Contents
Description
Symbolic music generation is a broad topic. It covers a wide range of tasks, including generation, harmonization, arrangement, instrumentation, and more. We have multiple ways to represent music data, and the evaluation metrics also vary. To define a MIREX challenge within this topic, we need to narrow our focus to specific subtasks that are both relevant to the community and feasible to evaluate effectively.
This year, we select the task to be **piano accompaniment arrangement from a lead sheet**. The lead sheet provides information about the melody, chord progression, and optional phrase labels. The goal is to generate a piano accompaniment that complements the lead melody. The music data consists of 8-measure segments in 4/4 meter, quantized to a sixteenth-note resolution. A more detailed description of the data structure is provided in the data format section. The genre of the lead sheets is broadly within western pop music (refer to the music examples for more detail).
Data Format
Evaluation
We will evaluate the submitted algorithms through an online subjective double-blind test.
- We use []. Before submitting the algorithm, each team is required to submit two lead sheets. The organizer team will supplement the lead sheet if necessary.
- Once the organizer team receives the lead sheet submission, a schedule for the subjective test will be made. Depending on the number of submissions, there could be more than one round of subjective test.
- We will use objective measurements only as a reference. The correlation between subjective and objective scores will be presented in the end.
Subjective Metrics
We will ask the subjects to listen to the original lead sheet first and then listen to the anonymized generations. Subjects will be asked to rate the generation according to the following criteria:
- Harmony correctness (5-point scale) (To be done)
- Creativity (5-point scale) (To be done)
- Naturalness (5-point scale) (To be done)
- To be done
Objective Metrics
We will report these metrics as reference:
- To be done.
