Difference between revisions of "2024:Symbolic Music Generation"
Zizzi wang (talk | contribs) (→Subjective Metrics) |
Zizzi wang (talk | contribs) (→Baselines) |
||
(8 intermediate revisions by the same user not shown) | |||
Line 2: | Line 2: | ||
Symbolic music generation is a broad topic. It covers a wide range of tasks, including generation, harmonization, arrangement, instrumentation, and more. We have multiple ways to represent music data, and the evaluation metrics also vary. To define a MIREX challenge within this topic, we need to narrow our focus to specific subtasks that are both relevant to the community and feasible to evaluate effectively. | Symbolic music generation is a broad topic. It covers a wide range of tasks, including generation, harmonization, arrangement, instrumentation, and more. We have multiple ways to represent music data, and the evaluation metrics also vary. To define a MIREX challenge within this topic, we need to narrow our focus to specific subtasks that are both relevant to the community and feasible to evaluate effectively. | ||
− | This year, we select the task to be | + | This year, we select the task to be '''piano accompaniment arrangement from a lead sheet'''. The lead sheet provides information about the melody, chord progression, and optional phrase labels. The goal is to generate a piano accompaniment that complements the lead melody. The music data consists of 8-measure segments in 4/4 meter, quantized to a sixteenth-note resolution. A more detailed description of the data structure is provided in the data format section. The genre of the lead sheets is broadly within western pop music (refer to the music examples for more detail). |
==Data Format== | ==Data Format== | ||
+ | The input lead sheet has three components: | ||
+ | * Melody: a list of (onset, pitch, duration) tuples. Onsets and durations are counted in sixteenth notes. Pitches are MIDI pitches from 0-127. | ||
+ | * Chord: a list of (onset, chord, duration) tuples. The chord contains root (ranging from 0-11), chroma (12-d multi-hot vector), and bass (ranging from 0-11) information. | ||
+ | * Phrase Label: TBD. | ||
+ | |||
+ | The lead sheet is 8 bars long and includes an additional pickup measure. The pickup measure ranges from time steps 0-15, and the eight measures range from time steps 16 to 143. The pickup measure is empty if there are no pickup beats. Notes ending later than the end of the eighth measure will be truncated to the end. | ||
+ | |||
+ | Output: | ||
+ | |||
+ | * Accompaniment: a list of (onset, pitch, duration) tuples. | ||
==Evaluation== | ==Evaluation== | ||
We will evaluate the submitted algorithms through an online subjective double-blind test. | We will evaluate the submitted algorithms through an online subjective double-blind test. | ||
− | * We use | + | * We use a "potluck" test set. Before submitting the algorithm, each team is required to submit two lead sheets. The organizer team will supplement the lead sheet if necessary. |
* Once the organizer team receives the lead sheet submission, a schedule for the subjective test will be made. Depending on the number of submissions, there could be more than one round of subjective test. | * Once the organizer team receives the lead sheet submission, a schedule for the subjective test will be made. Depending on the number of submissions, there could be more than one round of subjective test. | ||
* We will use objective measurements only as a reference. The correlation between subjective and objective scores will be presented in the end. | * We will use objective measurements only as a reference. The correlation between subjective and objective scores will be presented in the end. | ||
Line 18: | Line 28: | ||
* Creativity (5-point scale) (To be done) | * Creativity (5-point scale) (To be done) | ||
* Naturalness (5-point scale) (To be done) | * Naturalness (5-point scale) (To be done) | ||
− | * To be done | + | * Coherence and stylistic appropriateness? To be done. |
===Objective Metrics=== | ===Objective Metrics=== | ||
Line 25: | Line 35: | ||
==Data Example== | ==Data Example== | ||
+ | TBD | ||
+ | ==Submission== | ||
− | + | The submission process contains three stages. | |
+ | * [Date 1, TBA]: submission of two lead sheets as a part of the test set. This is also a confirmation of participation. The detailed subjective evaluation format will be announced according to the number of participants. | ||
+ | * Date 2: submission of code. [JJY can help.] | ||
+ | * Date 3: complete the online subjective evaluation. | ||
+ | ===Format=== | ||
+ | TBD. JJY can help. | ||
==Baselines== | ==Baselines== | ||
+ | TBD |
Latest revision as of 08:12, 25 August 2024
Contents
Description
Symbolic music generation is a broad topic. It covers a wide range of tasks, including generation, harmonization, arrangement, instrumentation, and more. We have multiple ways to represent music data, and the evaluation metrics also vary. To define a MIREX challenge within this topic, we need to narrow our focus to specific subtasks that are both relevant to the community and feasible to evaluate effectively.
This year, we select the task to be piano accompaniment arrangement from a lead sheet. The lead sheet provides information about the melody, chord progression, and optional phrase labels. The goal is to generate a piano accompaniment that complements the lead melody. The music data consists of 8-measure segments in 4/4 meter, quantized to a sixteenth-note resolution. A more detailed description of the data structure is provided in the data format section. The genre of the lead sheets is broadly within western pop music (refer to the music examples for more detail).
Data Format
The input lead sheet has three components:
- Melody: a list of (onset, pitch, duration) tuples. Onsets and durations are counted in sixteenth notes. Pitches are MIDI pitches from 0-127.
- Chord: a list of (onset, chord, duration) tuples. The chord contains root (ranging from 0-11), chroma (12-d multi-hot vector), and bass (ranging from 0-11) information.
- Phrase Label: TBD.
The lead sheet is 8 bars long and includes an additional pickup measure. The pickup measure ranges from time steps 0-15, and the eight measures range from time steps 16 to 143. The pickup measure is empty if there are no pickup beats. Notes ending later than the end of the eighth measure will be truncated to the end.
Output:
- Accompaniment: a list of (onset, pitch, duration) tuples.
Evaluation
We will evaluate the submitted algorithms through an online subjective double-blind test.
- We use a "potluck" test set. Before submitting the algorithm, each team is required to submit two lead sheets. The organizer team will supplement the lead sheet if necessary.
- Once the organizer team receives the lead sheet submission, a schedule for the subjective test will be made. Depending on the number of submissions, there could be more than one round of subjective test.
- We will use objective measurements only as a reference. The correlation between subjective and objective scores will be presented in the end.
Subjective Metrics
We will ask the subjects to listen to the original lead sheet first and then listen to the anonymized generations. Subjects will be asked to rate the generation according to the following criteria:
- Harmony correctness (5-point scale) (To be done)
- Creativity (5-point scale) (To be done)
- Naturalness (5-point scale) (To be done)
- Coherence and stylistic appropriateness? To be done.
Objective Metrics
We will report these metrics as reference:
- To be done.
Data Example
TBD
Submission
The submission process contains three stages.
- [Date 1, TBA]: submission of two lead sheets as a part of the test set. This is also a confirmation of participation. The detailed subjective evaluation format will be announced according to the number of participants.
- Date 2: submission of code. [JJY can help.]
- Date 3: complete the online subjective evaluation.
Format
TBD. JJY can help.
Baselines
TBD