MIREX Wiki - User contributions [en]

2026:Rhythm Game Chart Generation Results

2026-10-14T08:57:48Z

Carolina Carusi: /* Community Evaluation Results */

= Submissions =

{| class="wikitable"
|-
! Submission
! Title
! PDF
! Authors
|-
| DOSU
| DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/DOSU.pdf PDF]
| Junyoung Koh, Hao-Wen Dong
|-
| FumenLab
| FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/FumenLab.pdf PDF]
| Ryo Mitoma
|-
| HiTaiko
| HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/HiTaiko.pdf PDF]
| Sungjung Park
|-
| MALer
| RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/MALer.pdf PDF]
| Sojeong An, Chris Donahue, Dasaem Jeong
|}

= Test Set =

=== Song Selection ===

{| class="wikitable"
|-
! Title
! Artist
! Genre
! Language
! Tags
! BPM
|-
| Silver Bullet
| Ennnn
| Electronic
| Instrumental
| breakcore, neurofunk, tech
| 89-379 (mostly 175)
|-
| A fight in Tunisia
| Sobrem & NIWASHI
| Jazz
| Instrumental
| piano
| 148
|-
| Something Fun
| JOYLESS
| Rock
| English
| male vocal
| 194
|-
| closer to you (than ever before)
| xiiiac13
| Electronic
| Instrumental
| lo-fi
| 110
|-
| YUBIKIRI-GENMAN
| Mili
| Pop
| Japanese
| j-pop, progressive, female vocal
| 59-146 (mostly 146)
|-
| Himitsu feat. KAFU
| MIMI
| Pop
| Japanese
| j-pop, vocaloid
| 168
|}

=== Default Settings ===

{| class="wikitable"
|-
! Parameter
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Overall Difficulty
| 3
| 4
| 5
| 5
| 6
|-
| HP Drain Rate
| 8
| 7
| 6
| 5
| 5
|}

= Results =

== Algorithmic Evaluation Results ==

120 beatmaps were checked using [https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/AiMod AiMod] across 4 models, 6 songs, and 5 difficulties.

'''Difficulty ordering warnings:'''
* DOSU - closer to you - Inner Oni < Oni
* MALer - YUBIKIRI-GENMAN - Inner Oni < Oni
* MALer - closer to you - Kantan < Oni < Futsuu < Inner Oni < Muzukashii

=== DOSU ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu feat. KAFU
| OK
| OK
| OK
| OK
| OK
|-
| YUBIKIRI-GENMAN
| OK
| OK
| OK
| OK
| OK
|-
| Something Fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== FumenLab ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Objects <10ms apart and concurrent objects
| Objects <10ms apart and concurrent objects
|-
| A fight in Tunisia
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
| OK
|-
| Himitsu feat. KAFU
| OK
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
|-
| YUBIKIRI-GENMAN
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
|-
| Something Fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== HiTaiko ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu feat. KAFU
| OK
| OK
| OK
| OK
| OK
|-
| YUBIKIRI-GENMAN
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
|-
| Something Fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| OK
|}

=== MALer ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu feat. KAFU
| OK
| OK
| OK
| OK
| OK
|-
| YUBIKIRI-GENMAN
| OK
| OK
| OK
| OK
| OK
|-
| Something Fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
|}

== Expert Evaluation Results ==

5 osu!taiko mappers evaluated submissions across 5 songs: '''Silver Bullet''', '''A fight in Tunisia''', '''YUBIKIRI-GENMAN''', '''Something Fun''', and '''closer to you'''. Each judge was assigned 3 songs based on their familiarity and interest of the songs, and each song was evaluated by 3 judges. For every assigned song, judges assessed submissions from all 4 models on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10.

We follow [https://osu.ppy.sh/wiki/en/Contests/Monthly_Beatmapping_Contest Monthly Beatmapping Contest] and adopt '''within-song normalization''' for per-song report. For each category, judge X's final score = (judge X's raw score - judge X's average raw score) / judge X's standard deviation within that song, and a submission's score is the sum of each judge's final score.

=== Silver Bullet ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| '''1.5872'''
| −2.6304
| '''2.0099'''
| −0.4552
| Musical representation drew mixed assessments. Oni and Muzukashii showed effective sound selection, but Inner Oni was conservative early and uneven near the ending. Inconsistent special-object use and variable-BPM handling weakened coherence across difficulties.
|-
| FumenLab
| 0.3005
| 0.9401
| −3.5863
| −1.9478
| Oni and Inner Oni showed promising sound selection, but awkward bursts and near-overlapping notes compromised gameplay near the ending. Kantan was excessively sparse, while intermediate difficulties were comparatively plain. Repeated musical phrases received inconsistent treatment.
|-
| HiTaiko
| 1.2778
| '''1.8629'''
| '''2.0099'''
| '''4.1634'''
| Coherent musical representation and expressive color patterns were recurring strengths. Inner Oni made quieter passages disproportionately dense, while Oni was comparatively restrained. Futsuu and Muzukashii showed promising structure, although assessments of the lower-tier ending differed. Finisher placement remained inconsistent.
|-
| MALer
| −3.1656
| −0.1726
| −0.4335
| −1.7604
| Imaginative Oni patterns and distinctive sectioning were offset by rhythms that often felt disconnected from the music. Inner Oni contained timing and overlap problems; unusual subdivisions in Muzukashii and questionable sound selection in Futsuu weakened the difficulty spread.
|}

=== A fight in Tunisia ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −4.7037
| −3.6090
| 0.4677
| −1.5881
| The mapping frequently missed prominent musical events across all difficulties. Conservative patterns supported playability, but omitted piano passages and notes in musically empty sections weakened representation. Isolated Inner Oni patterns showed potential without fitting the song.
|-
| FumenLab
| 0.2720
| −0.2284
| −2.4438
| −0.1690
| Awkward bursts, excessive local density and neglected piano passages weakened musical clarity. Kantan–Muzukashii lacked coherent sound selection; Oni was sometimes too demanding for its tier. Inner Oni was relatively stronger but remained rhythmically awkward.
|-
| HiTaiko
| 2.0747
| 0.9776
| '''2.2563'''
| '''2.3546'''
| Oni stood out for coherent sound selection and structure. Inner Oni was more conservative and relied on abrupt doublets, while Kantan–Muzukashii often omitted material or mixed musical layers inconsistently. Piano representation and repeated special-object placement remained weaknesses.
|-
| MALer
| '''2.3570'''
| '''2.8598'''
| −0.2803
| −0.5975
| Expressive color patterns attracted praise, but overall assessments differed. Complex snapping and inconsistent musical-layer selection reduced readability, particularly in Kantan–Muzukashii. Inner Oni offered some musical representation, although its structure and rhythmic emphasis remained uneven.
|}

=== YUBIKIRI-GENMAN ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −1.6143
| '''0.6706'''
| −1.2916
| −1.5202
| Excessive rhythmic improvisation disrupted the song’s gentle character. Muzukashii, Oni and Inner Oni often emphasized verses over choruses, weakening emotional progression. Kantan’s simpler rhythms were more acceptable, but consistency, object placement and musical correspondence remained concerns.
|-
| FumenLab
| 1.3093
| 0.3325
| '''2.5865'''
| 0.6172
| Assessments of musical coherence differed. Muzukashii and Oni sometimes followed the vocals effectively, but repeated phrases received inconsistent treatment. Inner Oni’s complex climax disrupted the gentle atmosphere. Kantan was sparse, and hitsound volume and effect choices required refinement.
|-
| HiTaiko
| '''1.3093'''
| −0.2970
| 1.8815
| '''4.0095'''
| Oni and Inner Oni showed effective vocal following and a convincing build toward the final chorus, although some feedback found them too conservative. Muzukashii’s density allocation was inconsistent; Kantan and Futsuu offered little content. Special-object use and omissions remained concerns.
|-
| MALer
| −1.0043
| −0.7060
| −3.1764
| −3.1065
| Irregular snapping and abrupt density changes were poorly suited to the gentle vocal/piano arrangement. Muzukashii, Oni and Inner Oni contained isolated difficulty spikes, with Oni reportedly harder than Inner Oni. Some color patterns and brief rhythm choices worked but were not sustained.
|}
''Musical Representation scores for FumenLab and HiTaiko round to the same value; HiTaiko has the higher unrounded score.''

=== Something Fun ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −4.1362
| −3.0094
| −2.8967
| −3.9851
| Rhythmic choices and sound selection were poorly aligned with the music. Missing notes, excessive finishers and misplaced spinners weakened consistency. Both Muzukashii and Inner Oni contained passages whose musical intent was unclear.
|-
| FumenLab
| 1.7808
| 2.0497
| '''2.0645'''
| '''2.1521'''
| The set was broadly serviceable, but repetitive color patterns and inconsistent finishers limited expressiveness. Kantan and Futsuu were weak, while some Inner Oni streams added unnecessary complexity. Isolated passages also raised timing concerns.
|-
| HiTaiko
| −1.0707
| −1.3013
| −0.3393
| 0.6620
| Some patterns followed the melody effectively, but prolonged simple rhythms, questionable drumrolls and uneven sectional difficulty made the experience repetitive. Finisher placement lacked consistency; no particular difficulty tier emerged as a clear strength.
|-
| MALer
| '''3.4261'''
| '''2.2610'''
| 1.1716
| 1.1710
| Creative ideas produced sharply divided assessments. Futsuu omitted a conspicuous musical passage, while Inner Oni showed uneven density and timing problems. Kantan and Muzukashii had questionable rhythmic choices; Oni was comparatively more successful but appeared too easy for its label.
|}

=== closer to you ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −0.5579
| 0.7683
| −0.7678
| −0.5907
| Rhythmic improvisation showed promise, and Futsuu conveyed a pleasing feel. However, inconsistent finishers and uneven density weakened progression: Oni omitted the opening, while Inner Oni used fewer streams than Muzukashii.
|-
| FumenLab
| '''3.3983'''
| '''4.2866'''
| '''4.4877'''
| '''3.5659'''
| Strong structure and effective rhythmic improvisation were recurring positives, with Inner Oni receiving particular praise. Color-pattern variety, consistency and timing-point use still needed refinement, but the set conveyed a comparatively convincing mapping approach.
|-
| HiTaiko
| 1.6737
| −2.5275
| −0.1399
| 1.0151
| Patterns were coherent but repetitive, with little distinction between difficulty tiers. Even Inner Oni remained sparse and closer to a much easier difficulty. Drumroll use and the ending finisher were questionable choices for the song’s calm character.
|-
| MALer
| −4.5141
| −2.5275
| −3.5800
| −3.9903
| Missing sections, timing errors and inconsistent musical logic undermined the spread. Muzukashii was reportedly the hardest tier, while easier difficulties contained unexplained breaks. Inner Oni’s awkward fast bursts did not resolve the overall lack of musical coverage.
|}

Qualitative feedback suggested distinct tendencies across the evaluated models. HiTaiko often favored coherent but conservative arrangements, while MALer explored more unusual rhythms with less consistent musical fit. FumenLab demonstrated effective improvisation and higher-difficulty patterning, but struggled with simplification and technical consistency. DOSU showed isolated strengths in sound selection, alongside substantial variation in musical correspondence and structural coherence. Across models, adapting musical ideas to lower difficulties remained a recurring challenge.

== Community Evaluation Results ==

X osu!taiko players evaluated submissions on one test song: Himitsu feat. KAFU. Players assessed submissions from all 4 models, plus one beatmap created by a human mapper, on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10. They were not informed that any of the beatmaps were human-made. Every participant selected their preferred difficulty level and evaluated only the beatmaps of that difficulty.

=== Himitsu feat. KAFU ===
{| class="wikitable"
|-
! Mapper
! Musical Representation
! Creativity
! Gameplay
! Overall Quality Score
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
|
|
|
|
|
|
|-
| FumenLab
|
|
|
|
|
|
|-
| HiTaiko
|
|
|
|
|
|
|-
| MALer
|
|
|
|
|
|
|-
| Amasugi (human mapper)
|
|
|
|
|
|
|}

2026:Rhythm Game Chart Generation Results

2026-10-14T08:52:27Z

Carolina Carusi: /* Algorithmic Evaluation Results */

= Submissions =

{| class="wikitable"
|-
! Submission
! Title
! PDF
! Authors
|-
| DOSU
| DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/DOSU.pdf PDF]
| Junyoung Koh, Hao-Wen Dong
|-
| FumenLab
| FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/FumenLab.pdf PDF]
| Ryo Mitoma
|-
| HiTaiko
| HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/HiTaiko.pdf PDF]
| Sungjung Park
|-
| MALer
| RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/MALer.pdf PDF]
| Sojeong An, Chris Donahue, Dasaem Jeong
|}

= Test Set =

=== Song Selection ===

{| class="wikitable"
|-
! Title
! Artist
! Genre
! Language
! Tags
! BPM
|-
| Silver Bullet
| Ennnn
| Electronic
| Instrumental
| breakcore, neurofunk, tech
| 89-379 (mostly 175)
|-
| A fight in Tunisia
| Sobrem & NIWASHI
| Jazz
| Instrumental
| piano
| 148
|-
| Something Fun
| JOYLESS
| Rock
| English
| male vocal
| 194
|-
| closer to you (than ever before)
| xiiiac13
| Electronic
| Instrumental
| lo-fi
| 110
|-
| YUBIKIRI-GENMAN
| Mili
| Pop
| Japanese
| j-pop, progressive, female vocal
| 59-146 (mostly 146)
|-
| Himitsu feat. KAFU
| MIMI
| Pop
| Japanese
| j-pop, vocaloid
| 168
|}

=== Default Settings ===

{| class="wikitable"
|-
! Parameter
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Overall Difficulty
| 3
| 4
| 5
| 5
| 6
|-
| HP Drain Rate
| 8
| 7
| 6
| 5
| 5
|}

= Results =

== Algorithmic Evaluation Results ==

120 beatmaps were checked using [https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/AiMod AiMod] across 4 models, 6 songs, and 5 difficulties.

'''Difficulty ordering warnings:'''
* DOSU - closer to you - Inner Oni < Oni
* MALer - YUBIKIRI-GENMAN - Inner Oni < Oni
* MALer - closer to you - Kantan < Oni < Futsuu < Inner Oni < Muzukashii

=== DOSU ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu feat. KAFU
| OK
| OK
| OK
| OK
| OK
|-
| YUBIKIRI-GENMAN
| OK
| OK
| OK
| OK
| OK
|-
| Something Fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== FumenLab ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Objects <10ms apart and concurrent objects
| Objects <10ms apart and concurrent objects
|-
| A fight in Tunisia
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
| OK
|-
| Himitsu feat. KAFU
| OK
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
|-
| YUBIKIRI-GENMAN
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
|-
| Something Fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== HiTaiko ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu feat. KAFU
| OK
| OK
| OK
| OK
| OK
|-
| YUBIKIRI-GENMAN
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
|-
| Something Fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| OK
|}

=== MALer ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu feat. KAFU
| OK
| OK
| OK
| OK
| OK
|-
| YUBIKIRI-GENMAN
| OK
| OK
| OK
| OK
| OK
|-
| Something Fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
|}

== Expert Evaluation Results ==

5 osu!taiko mappers evaluated submissions across 5 songs: '''Silver Bullet''', '''A fight in Tunisia''', '''YUBIKIRI-GENMAN''', '''Something Fun''', and '''closer to you'''. Each judge was assigned 3 songs based on their familiarity and interest of the songs, and each song was evaluated by 3 judges. For every assigned song, judges assessed submissions from all 4 models on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10.

We follow [https://osu.ppy.sh/wiki/en/Contests/Monthly_Beatmapping_Contest Monthly Beatmapping Contest] and adopt '''within-song normalization''' for per-song report. For each category, judge X's final score = (judge X's raw score - judge X's average raw score) / judge X's standard deviation within that song, and a submission's score is the sum of each judge's final score.

=== Silver Bullet ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| '''1.5872'''
| −2.6304
| '''2.0099'''
| −0.4552
| Musical representation drew mixed assessments. Oni and Muzukashii showed effective sound selection, but Inner Oni was conservative early and uneven near the ending. Inconsistent special-object use and variable-BPM handling weakened coherence across difficulties.
|-
| FumenLab
| 0.3005
| 0.9401
| −3.5863
| −1.9478
| Oni and Inner Oni showed promising sound selection, but awkward bursts and near-overlapping notes compromised gameplay near the ending. Kantan was excessively sparse, while intermediate difficulties were comparatively plain. Repeated musical phrases received inconsistent treatment.
|-
| HiTaiko
| 1.2778
| '''1.8629'''
| '''2.0099'''
| '''4.1634'''
| Coherent musical representation and expressive color patterns were recurring strengths. Inner Oni made quieter passages disproportionately dense, while Oni was comparatively restrained. Futsuu and Muzukashii showed promising structure, although assessments of the lower-tier ending differed. Finisher placement remained inconsistent.
|-
| MALer
| −3.1656
| −0.1726
| −0.4335
| −1.7604
| Imaginative Oni patterns and distinctive sectioning were offset by rhythms that often felt disconnected from the music. Inner Oni contained timing and overlap problems; unusual subdivisions in Muzukashii and questionable sound selection in Futsuu weakened the difficulty spread.
|}

=== A fight in Tunisia ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −4.7037
| −3.6090
| 0.4677
| −1.5881
| The mapping frequently missed prominent musical events across all difficulties. Conservative patterns supported playability, but omitted piano passages and notes in musically empty sections weakened representation. Isolated Inner Oni patterns showed potential without fitting the song.
|-
| FumenLab
| 0.2720
| −0.2284
| −2.4438
| −0.1690
| Awkward bursts, excessive local density and neglected piano passages weakened musical clarity. Kantan–Muzukashii lacked coherent sound selection; Oni was sometimes too demanding for its tier. Inner Oni was relatively stronger but remained rhythmically awkward.
|-
| HiTaiko
| 2.0747
| 0.9776
| '''2.2563'''
| '''2.3546'''
| Oni stood out for coherent sound selection and structure. Inner Oni was more conservative and relied on abrupt doublets, while Kantan–Muzukashii often omitted material or mixed musical layers inconsistently. Piano representation and repeated special-object placement remained weaknesses.
|-
| MALer
| '''2.3570'''
| '''2.8598'''
| −0.2803
| −0.5975
| Expressive color patterns attracted praise, but overall assessments differed. Complex snapping and inconsistent musical-layer selection reduced readability, particularly in Kantan–Muzukashii. Inner Oni offered some musical representation, although its structure and rhythmic emphasis remained uneven.
|}

=== YUBIKIRI-GENMAN ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −1.6143
| '''0.6706'''
| −1.2916
| −1.5202
| Excessive rhythmic improvisation disrupted the song’s gentle character. Muzukashii, Oni and Inner Oni often emphasized verses over choruses, weakening emotional progression. Kantan’s simpler rhythms were more acceptable, but consistency, object placement and musical correspondence remained concerns.
|-
| FumenLab
| 1.3093
| 0.3325
| '''2.5865'''
| 0.6172
| Assessments of musical coherence differed. Muzukashii and Oni sometimes followed the vocals effectively, but repeated phrases received inconsistent treatment. Inner Oni’s complex climax disrupted the gentle atmosphere. Kantan was sparse, and hitsound volume and effect choices required refinement.
|-
| HiTaiko
| '''1.3093'''
| −0.2970
| 1.8815
| '''4.0095'''
| Oni and Inner Oni showed effective vocal following and a convincing build toward the final chorus, although some feedback found them too conservative. Muzukashii’s density allocation was inconsistent; Kantan and Futsuu offered little content. Special-object use and omissions remained concerns.
|-
| MALer
| −1.0043
| −0.7060
| −3.1764
| −3.1065
| Irregular snapping and abrupt density changes were poorly suited to the gentle vocal/piano arrangement. Muzukashii, Oni and Inner Oni contained isolated difficulty spikes, with Oni reportedly harder than Inner Oni. Some color patterns and brief rhythm choices worked but were not sustained.
|}
''Musical Representation scores for FumenLab and HiTaiko round to the same value; HiTaiko has the higher unrounded score.''

=== Something Fun ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −4.1362
| −3.0094
| −2.8967
| −3.9851
| Rhythmic choices and sound selection were poorly aligned with the music. Missing notes, excessive finishers and misplaced spinners weakened consistency. Both Muzukashii and Inner Oni contained passages whose musical intent was unclear.
|-
| FumenLab
| 1.7808
| 2.0497
| '''2.0645'''
| '''2.1521'''
| The set was broadly serviceable, but repetitive color patterns and inconsistent finishers limited expressiveness. Kantan and Futsuu were weak, while some Inner Oni streams added unnecessary complexity. Isolated passages also raised timing concerns.
|-
| HiTaiko
| −1.0707
| −1.3013
| −0.3393
| 0.6620
| Some patterns followed the melody effectively, but prolonged simple rhythms, questionable drumrolls and uneven sectional difficulty made the experience repetitive. Finisher placement lacked consistency; no particular difficulty tier emerged as a clear strength.
|-
| MALer
| '''3.4261'''
| '''2.2610'''
| 1.1716
| 1.1710
| Creative ideas produced sharply divided assessments. Futsuu omitted a conspicuous musical passage, while Inner Oni showed uneven density and timing problems. Kantan and Muzukashii had questionable rhythmic choices; Oni was comparatively more successful but appeared too easy for its label.
|}

=== closer to you ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −0.5579
| 0.7683
| −0.7678
| −0.5907
| Rhythmic improvisation showed promise, and Futsuu conveyed a pleasing feel. However, inconsistent finishers and uneven density weakened progression: Oni omitted the opening, while Inner Oni used fewer streams than Muzukashii.
|-
| FumenLab
| '''3.3983'''
| '''4.2866'''
| '''4.4877'''
| '''3.5659'''
| Strong structure and effective rhythmic improvisation were recurring positives, with Inner Oni receiving particular praise. Color-pattern variety, consistency and timing-point use still needed refinement, but the set conveyed a comparatively convincing mapping approach.
|-
| HiTaiko
| 1.6737
| −2.5275
| −0.1399
| 1.0151
| Patterns were coherent but repetitive, with little distinction between difficulty tiers. Even Inner Oni remained sparse and closer to a much easier difficulty. Drumroll use and the ending finisher were questionable choices for the song’s calm character.
|-
| MALer
| −4.5141
| −2.5275
| −3.5800
| −3.9903
| Missing sections, timing errors and inconsistent musical logic undermined the spread. Muzukashii was reportedly the hardest tier, while easier difficulties contained unexplained breaks. Inner Oni’s awkward fast bursts did not resolve the overall lack of musical coverage.
|}

Qualitative feedback suggested distinct tendencies across the evaluated models. HiTaiko often favored coherent but conservative arrangements, while MALer explored more unusual rhythms with less consistent musical fit. FumenLab demonstrated effective improvisation and higher-difficulty patterning, but struggled with simplification and technical consistency. DOSU showed isolated strengths in sound selection, alongside substantial variation in musical correspondence and structural coherence. Across models, adapting musical ideas to lower difficulties remained a recurring challenge.

== Community Evaluation Results ==

X osu!taiko players evaluated submissions on one test song: Himitsu feat. KAFU. Players assessed submissions from all 4 models, plus one beatmap created by a human mapper, on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10. They were not informed which beatmap was human-made. Every participant selected their preferred difficulty level and evaluated only the beatmaps of that difficulty.

=== Himitsu feat. KAFU ===
{| class="wikitable"
|-
! Mapper
! Musical Representation
! Creativity
! Gameplay
! Overall Quality Score
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
|
|
|
|
|
|
|-
| FumenLab
|
|
|
|
|
|
|-
| HiTaiko
|
|
|
|
|
|
|-
| MALer
|
|
|
|
|
|
|-
| Amasugi (human mapper)
|
|
|
|
|
|
|}

2026:Rhythm Game Chart Generation Results

2026-10-14T08:51:21Z

Carolina Carusi: /* Algorithmic Evaluation Results */

= Submissions =

{| class="wikitable"
|-
! Submission
! Title
! PDF
! Authors
|-
| DOSU
| DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/DOSU.pdf PDF]
| Junyoung Koh, Hao-Wen Dong
|-
| FumenLab
| FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/FumenLab.pdf PDF]
| Ryo Mitoma
|-
| HiTaiko
| HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/HiTaiko.pdf PDF]
| Sungjung Park
|-
| MALer
| RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/MALer.pdf PDF]
| Sojeong An, Chris Donahue, Dasaem Jeong
|}

= Test Set =

=== Song Selection ===

{| class="wikitable"
|-
! Title
! Artist
! Genre
! Language
! Tags
! BPM
|-
| Silver Bullet
| Ennnn
| Electronic
| Instrumental
| breakcore, neurofunk, tech
| 89-379 (mostly 175)
|-
| A fight in Tunisia
| Sobrem & NIWASHI
| Jazz
| Instrumental
| piano
| 148
|-
| Something Fun
| JOYLESS
| Rock
| English
| male vocal
| 194
|-
| closer to you (than ever before)
| xiiiac13
| Electronic
| Instrumental
| lo-fi
| 110
|-
| YUBIKIRI-GENMAN
| Mili
| Pop
| Japanese
| j-pop, progressive, female vocal
| 59-146 (mostly 146)
|-
| Himitsu feat. KAFU
| MIMI
| Pop
| Japanese
| j-pop, vocaloid
| 168
|}

=== Default Settings ===

{| class="wikitable"
|-
! Parameter
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Overall Difficulty
| 3
| 4
| 5
| 5
| 6
|-
| HP Drain Rate
| 8
| 7
| 6
| 5
| 5
|}

= Results =

== Algorithmic Evaluation Results ==

120 beatmaps were checked using [https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/AiMod AiMod] across 4 models, 6 songs, and 5 difficulties.

'''Difficulty ordering warnings:'''
* DOSU - closer to you - Inner Oni < Oni
* MALer - YUBIKIRI-GENMAN - Inner Oni < Oni
* MALer - closer to you - Kantan < Oni < Futsuu < Inner Oni < Muzukashii

=== DOSU ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu feat. KAFU
| OK
| OK
| OK
| OK
| OK
|-
| YUBIKIRI-GENMAN
| OK
| OK
| OK
| OK
| OK
|-
| Something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== FumenLab ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Objects <10ms apart and concurrent objects
| Objects <10ms apart and concurrent objects
|-
| A fight in Tunisia
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
| OK
|-
| Himitsu feat. KAFU
| OK
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
|-
| YUBIKIRI-GENMAN
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
|-
| Something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== HiTaiko ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu feat. KAFU
| OK
| OK
| OK
| OK
| OK
|-
| YUBIKIRI-GENMAN
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
|-
| Something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| OK
|}

=== MALer ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu feat. KAFU
| OK
| OK
| OK
| OK
| OK
|-
| YUBIKIRI-GENMAN
| OK
| OK
| OK
| OK
| OK
|-
| Something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
|}

== Expert Evaluation Results ==

5 osu!taiko mappers evaluated submissions across 5 songs: '''Silver Bullet''', '''A fight in Tunisia''', '''YUBIKIRI-GENMAN''', '''Something Fun''', and '''closer to you'''. Each judge was assigned 3 songs based on their familiarity and interest of the songs, and each song was evaluated by 3 judges. For every assigned song, judges assessed submissions from all 4 models on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10.

We follow [https://osu.ppy.sh/wiki/en/Contests/Monthly_Beatmapping_Contest Monthly Beatmapping Contest] and adopt '''within-song normalization''' for per-song report. For each category, judge X's final score = (judge X's raw score - judge X's average raw score) / judge X's standard deviation within that song, and a submission's score is the sum of each judge's final score.

=== Silver Bullet ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| '''1.5872'''
| −2.6304
| '''2.0099'''
| −0.4552
| Musical representation drew mixed assessments. Oni and Muzukashii showed effective sound selection, but Inner Oni was conservative early and uneven near the ending. Inconsistent special-object use and variable-BPM handling weakened coherence across difficulties.
|-
| FumenLab
| 0.3005
| 0.9401
| −3.5863
| −1.9478
| Oni and Inner Oni showed promising sound selection, but awkward bursts and near-overlapping notes compromised gameplay near the ending. Kantan was excessively sparse, while intermediate difficulties were comparatively plain. Repeated musical phrases received inconsistent treatment.
|-
| HiTaiko
| 1.2778
| '''1.8629'''
| '''2.0099'''
| '''4.1634'''
| Coherent musical representation and expressive color patterns were recurring strengths. Inner Oni made quieter passages disproportionately dense, while Oni was comparatively restrained. Futsuu and Muzukashii showed promising structure, although assessments of the lower-tier ending differed. Finisher placement remained inconsistent.
|-
| MALer
| −3.1656
| −0.1726
| −0.4335
| −1.7604
| Imaginative Oni patterns and distinctive sectioning were offset by rhythms that often felt disconnected from the music. Inner Oni contained timing and overlap problems; unusual subdivisions in Muzukashii and questionable sound selection in Futsuu weakened the difficulty spread.
|}

=== A fight in Tunisia ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −4.7037
| −3.6090
| 0.4677
| −1.5881
| The mapping frequently missed prominent musical events across all difficulties. Conservative patterns supported playability, but omitted piano passages and notes in musically empty sections weakened representation. Isolated Inner Oni patterns showed potential without fitting the song.
|-
| FumenLab
| 0.2720
| −0.2284
| −2.4438
| −0.1690
| Awkward bursts, excessive local density and neglected piano passages weakened musical clarity. Kantan–Muzukashii lacked coherent sound selection; Oni was sometimes too demanding for its tier. Inner Oni was relatively stronger but remained rhythmically awkward.
|-
| HiTaiko
| 2.0747
| 0.9776
| '''2.2563'''
| '''2.3546'''
| Oni stood out for coherent sound selection and structure. Inner Oni was more conservative and relied on abrupt doublets, while Kantan–Muzukashii often omitted material or mixed musical layers inconsistently. Piano representation and repeated special-object placement remained weaknesses.
|-
| MALer
| '''2.3570'''
| '''2.8598'''
| −0.2803
| −0.5975
| Expressive color patterns attracted praise, but overall assessments differed. Complex snapping and inconsistent musical-layer selection reduced readability, particularly in Kantan–Muzukashii. Inner Oni offered some musical representation, although its structure and rhythmic emphasis remained uneven.
|}

=== YUBIKIRI-GENMAN ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −1.6143
| '''0.6706'''
| −1.2916
| −1.5202
| Excessive rhythmic improvisation disrupted the song’s gentle character. Muzukashii, Oni and Inner Oni often emphasized verses over choruses, weakening emotional progression. Kantan’s simpler rhythms were more acceptable, but consistency, object placement and musical correspondence remained concerns.
|-
| FumenLab
| 1.3093
| 0.3325
| '''2.5865'''
| 0.6172
| Assessments of musical coherence differed. Muzukashii and Oni sometimes followed the vocals effectively, but repeated phrases received inconsistent treatment. Inner Oni’s complex climax disrupted the gentle atmosphere. Kantan was sparse, and hitsound volume and effect choices required refinement.
|-
| HiTaiko
| '''1.3093'''
| −0.2970
| 1.8815
| '''4.0095'''
| Oni and Inner Oni showed effective vocal following and a convincing build toward the final chorus, although some feedback found them too conservative. Muzukashii’s density allocation was inconsistent; Kantan and Futsuu offered little content. Special-object use and omissions remained concerns.
|-
| MALer
| −1.0043
| −0.7060
| −3.1764
| −3.1065
| Irregular snapping and abrupt density changes were poorly suited to the gentle vocal/piano arrangement. Muzukashii, Oni and Inner Oni contained isolated difficulty spikes, with Oni reportedly harder than Inner Oni. Some color patterns and brief rhythm choices worked but were not sustained.
|}
''Musical Representation scores for FumenLab and HiTaiko round to the same value; HiTaiko has the higher unrounded score.''

=== Something Fun ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −4.1362
| −3.0094
| −2.8967
| −3.9851
| Rhythmic choices and sound selection were poorly aligned with the music. Missing notes, excessive finishers and misplaced spinners weakened consistency. Both Muzukashii and Inner Oni contained passages whose musical intent was unclear.
|-
| FumenLab
| 1.7808
| 2.0497
| '''2.0645'''
| '''2.1521'''
| The set was broadly serviceable, but repetitive color patterns and inconsistent finishers limited expressiveness. Kantan and Futsuu were weak, while some Inner Oni streams added unnecessary complexity. Isolated passages also raised timing concerns.
|-
| HiTaiko
| −1.0707
| −1.3013
| −0.3393
| 0.6620
| Some patterns followed the melody effectively, but prolonged simple rhythms, questionable drumrolls and uneven sectional difficulty made the experience repetitive. Finisher placement lacked consistency; no particular difficulty tier emerged as a clear strength.
|-
| MALer
| '''3.4261'''
| '''2.2610'''
| 1.1716
| 1.1710
| Creative ideas produced sharply divided assessments. Futsuu omitted a conspicuous musical passage, while Inner Oni showed uneven density and timing problems. Kantan and Muzukashii had questionable rhythmic choices; Oni was comparatively more successful but appeared too easy for its label.
|}

=== closer to you ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −0.5579
| 0.7683
| −0.7678
| −0.5907
| Rhythmic improvisation showed promise, and Futsuu conveyed a pleasing feel. However, inconsistent finishers and uneven density weakened progression: Oni omitted the opening, while Inner Oni used fewer streams than Muzukashii.
|-
| FumenLab
| '''3.3983'''
| '''4.2866'''
| '''4.4877'''
| '''3.5659'''
| Strong structure and effective rhythmic improvisation were recurring positives, with Inner Oni receiving particular praise. Color-pattern variety, consistency and timing-point use still needed refinement, but the set conveyed a comparatively convincing mapping approach.
|-
| HiTaiko
| 1.6737
| −2.5275
| −0.1399
| 1.0151
| Patterns were coherent but repetitive, with little distinction between difficulty tiers. Even Inner Oni remained sparse and closer to a much easier difficulty. Drumroll use and the ending finisher were questionable choices for the song’s calm character.
|-
| MALer
| −4.5141
| −2.5275
| −3.5800
| −3.9903
| Missing sections, timing errors and inconsistent musical logic undermined the spread. Muzukashii was reportedly the hardest tier, while easier difficulties contained unexplained breaks. Inner Oni’s awkward fast bursts did not resolve the overall lack of musical coverage.
|}

Qualitative feedback suggested distinct tendencies across the evaluated models. HiTaiko often favored coherent but conservative arrangements, while MALer explored more unusual rhythms with less consistent musical fit. FumenLab demonstrated effective improvisation and higher-difficulty patterning, but struggled with simplification and technical consistency. DOSU showed isolated strengths in sound selection, alongside substantial variation in musical correspondence and structural coherence. Across models, adapting musical ideas to lower difficulties remained a recurring challenge.

== Community Evaluation Results ==

X osu!taiko players evaluated submissions on one test song: Himitsu feat. KAFU. Players assessed submissions from all 4 models, plus one beatmap created by a human mapper, on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10. They were not informed which beatmap was human-made. Every participant selected their preferred difficulty level and evaluated only the beatmaps of that difficulty.

=== Himitsu feat. KAFU ===
{| class="wikitable"
|-
! Mapper
! Musical Representation
! Creativity
! Gameplay
! Overall Quality Score
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
|
|
|
|
|
|
|-
| FumenLab
|
|
|
|
|
|
|-
| HiTaiko
|
|
|
|
|
|
|-
| MALer
|
|
|
|
|
|
|-
| Amasugi (human mapper)
|
|
|
|
|
|
|}

2026:Rhythm Game Chart Generation Results

2026-10-14T08:48:29Z

Carolina Carusi: /* Community Evaluation Results */

= Submissions =

{| class="wikitable"
|-
! Submission
! Title
! PDF
! Authors
|-
| DOSU
| DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/DOSU.pdf PDF]
| Junyoung Koh, Hao-Wen Dong
|-
| FumenLab
| FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/FumenLab.pdf PDF]
| Ryo Mitoma
|-
| HiTaiko
| HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/HiTaiko.pdf PDF]
| Sungjung Park
|-
| MALer
| RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/MALer.pdf PDF]
| Sojeong An, Chris Donahue, Dasaem Jeong
|}

= Test Set =

=== Song Selection ===

{| class="wikitable"
|-
! Title
! Artist
! Genre
! Language
! Tags
! BPM
|-
| Silver Bullet
| Ennnn
| Electronic
| Instrumental
| breakcore, neurofunk, tech
| 89-379 (mostly 175)
|-
| A fight in Tunisia
| Sobrem & NIWASHI
| Jazz
| Instrumental
| piano
| 148
|-
| Something Fun
| JOYLESS
| Rock
| English
| male vocal
| 194
|-
| closer to you (than ever before)
| xiiiac13
| Electronic
| Instrumental
| lo-fi
| 110
|-
| YUBIKIRI-GENMAN
| Mili
| Pop
| Japanese
| j-pop, progressive, female vocal
| 59-146 (mostly 146)
|-
| Himitsu feat. KAFU
| MIMI
| Pop
| Japanese
| j-pop, vocaloid
| 168
|}

=== Default Settings ===

{| class="wikitable"
|-
! Parameter
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Overall Difficulty
| 3
| 4
| 5
| 5
| 6
|-
| HP Drain Rate
| 8
| 7
| 6
| 5
| 5
|}

= Results =

== Algorithmic Evaluation Results ==

120 beatmaps were checked using [https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/AiMod AiMod] across 4 models, 6 songs, and 5 difficulties.

'''Difficulty ordering warnings:'''
* DOSU - closer to you - Inner Oni < Oni
* MALer - YUBIKIRI-GENMAN - Inner Oni < Oni
* MALer - closer to you - Kantan < Oni < Futsuu < Inner Oni < Muzukashii

=== DOSU ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== FumenLab ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Objects <10ms apart and concurrent objects
| Objects <10ms apart and concurrent objects
|-
| A fight in Tunisia
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
|-
| yubikiri-genman
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== HiTaiko ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| OK
|}

=== MALer ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
|}

== Expert Evaluation Results ==

5 osu!taiko mappers evaluated submissions across 5 songs: '''Silver Bullet''', '''A fight in Tunisia''', '''YUBIKIRI-GENMAN''', '''Something Fun''', and '''closer to you'''. Each judge was assigned 3 songs based on their familiarity and interest of the songs, and each song was evaluated by 3 judges. For every assigned song, judges assessed submissions from all 4 models on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10.

We follow [https://osu.ppy.sh/wiki/en/Contests/Monthly_Beatmapping_Contest Monthly Beatmapping Contest] and adopt '''within-song normalization''' for per-song report. For each category, judge X's final score = (judge X's raw score - judge X's average raw score) / judge X's standard deviation within that song, and a submission's score is the sum of each judge's final score.

=== Silver Bullet ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| '''1.5872'''
| −2.6304
| '''2.0099'''
| −0.4552
| Musical representation drew mixed assessments. Oni and Muzukashii showed effective sound selection, but Inner Oni was conservative early and uneven near the ending. Inconsistent special-object use and variable-BPM handling weakened coherence across difficulties.
|-
| FumenLab
| 0.3005
| 0.9401
| −3.5863
| −1.9478
| Oni and Inner Oni showed promising sound selection, but awkward bursts and near-overlapping notes compromised gameplay near the ending. Kantan was excessively sparse, while intermediate difficulties were comparatively plain. Repeated musical phrases received inconsistent treatment.
|-
| HiTaiko
| 1.2778
| '''1.8629'''
| '''2.0099'''
| '''4.1634'''
| Coherent musical representation and expressive color patterns were recurring strengths. Inner Oni made quieter passages disproportionately dense, while Oni was comparatively restrained. Futsuu and Muzukashii showed promising structure, although assessments of the lower-tier ending differed. Finisher placement remained inconsistent.
|-
| MALer
| −3.1656
| −0.1726
| −0.4335
| −1.7604
| Imaginative Oni patterns and distinctive sectioning were offset by rhythms that often felt disconnected from the music. Inner Oni contained timing and overlap problems; unusual subdivisions in Muzukashii and questionable sound selection in Futsuu weakened the difficulty spread.
|}

=== A fight in Tunisia ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −4.7037
| −3.6090
| 0.4677
| −1.5881
| The mapping frequently missed prominent musical events across all difficulties. Conservative patterns supported playability, but omitted piano passages and notes in musically empty sections weakened representation. Isolated Inner Oni patterns showed potential without fitting the song.
|-
| FumenLab
| 0.2720
| −0.2284
| −2.4438
| −0.1690
| Awkward bursts, excessive local density and neglected piano passages weakened musical clarity. Kantan–Muzukashii lacked coherent sound selection; Oni was sometimes too demanding for its tier. Inner Oni was relatively stronger but remained rhythmically awkward.
|-
| HiTaiko
| 2.0747
| 0.9776
| '''2.2563'''
| '''2.3546'''
| Oni stood out for coherent sound selection and structure. Inner Oni was more conservative and relied on abrupt doublets, while Kantan–Muzukashii often omitted material or mixed musical layers inconsistently. Piano representation and repeated special-object placement remained weaknesses.
|-
| MALer
| '''2.3570'''
| '''2.8598'''
| −0.2803
| −0.5975
| Expressive color patterns attracted praise, but overall assessments differed. Complex snapping and inconsistent musical-layer selection reduced readability, particularly in Kantan–Muzukashii. Inner Oni offered some musical representation, although its structure and rhythmic emphasis remained uneven.
|}

=== YUBIKIRI-GENMAN ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −1.6143
| '''0.6706'''
| −1.2916
| −1.5202
| Excessive rhythmic improvisation disrupted the song’s gentle character. Muzukashii, Oni and Inner Oni often emphasized verses over choruses, weakening emotional progression. Kantan’s simpler rhythms were more acceptable, but consistency, object placement and musical correspondence remained concerns.
|-
| FumenLab
| 1.3093
| 0.3325
| '''2.5865'''
| 0.6172
| Assessments of musical coherence differed. Muzukashii and Oni sometimes followed the vocals effectively, but repeated phrases received inconsistent treatment. Inner Oni’s complex climax disrupted the gentle atmosphere. Kantan was sparse, and hitsound volume and effect choices required refinement.
|-
| HiTaiko
| '''1.3093'''
| −0.2970
| 1.8815
| '''4.0095'''
| Oni and Inner Oni showed effective vocal following and a convincing build toward the final chorus, although some feedback found them too conservative. Muzukashii’s density allocation was inconsistent; Kantan and Futsuu offered little content. Special-object use and omissions remained concerns.
|-
| MALer
| −1.0043
| −0.7060
| −3.1764
| −3.1065
| Irregular snapping and abrupt density changes were poorly suited to the gentle vocal/piano arrangement. Muzukashii, Oni and Inner Oni contained isolated difficulty spikes, with Oni reportedly harder than Inner Oni. Some color patterns and brief rhythm choices worked but were not sustained.
|}
''Musical Representation scores for FumenLab and HiTaiko round to the same value; HiTaiko has the higher unrounded score.''

=== Something Fun ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −4.1362
| −3.0094
| −2.8967
| −3.9851
| Rhythmic choices and sound selection were poorly aligned with the music. Missing notes, excessive finishers and misplaced spinners weakened consistency. Both Muzukashii and Inner Oni contained passages whose musical intent was unclear.
|-
| FumenLab
| 1.7808
| 2.0497
| '''2.0645'''
| '''2.1521'''
| The set was broadly serviceable, but repetitive color patterns and inconsistent finishers limited expressiveness. Kantan and Futsuu were weak, while some Inner Oni streams added unnecessary complexity. Isolated passages also raised timing concerns.
|-
| HiTaiko
| −1.0707
| −1.3013
| −0.3393
| 0.6620
| Some patterns followed the melody effectively, but prolonged simple rhythms, questionable drumrolls and uneven sectional difficulty made the experience repetitive. Finisher placement lacked consistency; no particular difficulty tier emerged as a clear strength.
|-
| MALer
| '''3.4261'''
| '''2.2610'''
| 1.1716
| 1.1710
| Creative ideas produced sharply divided assessments. Futsuu omitted a conspicuous musical passage, while Inner Oni showed uneven density and timing problems. Kantan and Muzukashii had questionable rhythmic choices; Oni was comparatively more successful but appeared too easy for its label.
|}

=== closer to you ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −0.5579
| 0.7683
| −0.7678
| −0.5907
| Rhythmic improvisation showed promise, and Futsuu conveyed a pleasing feel. However, inconsistent finishers and uneven density weakened progression: Oni omitted the opening, while Inner Oni used fewer streams than Muzukashii.
|-
| FumenLab
| '''3.3983'''
| '''4.2866'''
| '''4.4877'''
| '''3.5659'''
| Strong structure and effective rhythmic improvisation were recurring positives, with Inner Oni receiving particular praise. Color-pattern variety, consistency and timing-point use still needed refinement, but the set conveyed a comparatively convincing mapping approach.
|-
| HiTaiko
| 1.6737
| −2.5275
| −0.1399
| 1.0151
| Patterns were coherent but repetitive, with little distinction between difficulty tiers. Even Inner Oni remained sparse and closer to a much easier difficulty. Drumroll use and the ending finisher were questionable choices for the song’s calm character.
|-
| MALer
| −4.5141
| −2.5275
| −3.5800
| −3.9903
| Missing sections, timing errors and inconsistent musical logic undermined the spread. Muzukashii was reportedly the hardest tier, while easier difficulties contained unexplained breaks. Inner Oni’s awkward fast bursts did not resolve the overall lack of musical coverage.
|}

Qualitative feedback suggested distinct tendencies across the evaluated models. HiTaiko often favored coherent but conservative arrangements, while MALer explored more unusual rhythms with less consistent musical fit. FumenLab demonstrated effective improvisation and higher-difficulty patterning, but struggled with simplification and technical consistency. DOSU showed isolated strengths in sound selection, alongside substantial variation in musical correspondence and structural coherence. Across models, adapting musical ideas to lower difficulties remained a recurring challenge.

== Community Evaluation Results ==

X osu!taiko players evaluated submissions on one test song: Himitsu feat. KAFU. Players assessed submissions from all 4 models, plus one beatmap created by a human mapper, on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10. They were not informed which beatmap was human-made. Every participant selected their preferred difficulty level and evaluated only the beatmaps of that difficulty.

=== Himitsu feat. KAFU ===
{| class="wikitable"
|-
! Mapper
! Musical Representation
! Creativity
! Gameplay
! Overall Quality Score
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
|
|
|
|
|
|
|-
| FumenLab
|
|
|
|
|
|
|-
| HiTaiko
|
|
|
|
|
|
|-
| MALer
|
|
|
|
|
|
|-
| Amasugi (human mapper)
|
|
|
|
|
|
|}

2026:Rhythm Game Chart Generation Results

2026-10-14T08:34:13Z

Carolina Carusi: /* Community Evaluation Results */

= Submissions =

{| class="wikitable"
|-
! Submission
! Title
! PDF
! Authors
|-
| DOSU
| DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/DOSU.pdf PDF]
| Junyoung Koh, Hao-Wen Dong
|-
| FumenLab
| FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/FumenLab.pdf PDF]
| Ryo Mitoma
|-
| HiTaiko
| HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/HiTaiko.pdf PDF]
| Sungjung Park
|-
| MALer
| RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/MALer.pdf PDF]
| Sojeong An, Chris Donahue, Dasaem Jeong
|}

= Test Set =

=== Song Selection ===

{| class="wikitable"
|-
! Title
! Artist
! Genre
! Language
! Tags
! BPM
|-
| Silver Bullet
| Ennnn
| Electronic
| Instrumental
| breakcore, neurofunk, tech
| 89-379 (mostly 175)
|-
| A fight in Tunisia
| Sobrem & NIWASHI
| Jazz
| Instrumental
| piano
| 148
|-
| Something Fun
| JOYLESS
| Rock
| English
| male vocal
| 194
|-
| closer to you (than ever before)
| xiiiac13
| Electronic
| Instrumental
| lo-fi
| 110
|-
| YUBIKIRI-GENMAN
| Mili
| Pop
| Japanese
| j-pop, progressive, female vocal
| 59-146 (mostly 146)
|-
| Himitsu feat. KAFU
| MIMI
| Pop
| Japanese
| j-pop, vocaloid
| 168
|}

=== Default Settings ===

{| class="wikitable"
|-
! Parameter
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Overall Difficulty
| 3
| 4
| 5
| 5
| 6
|-
| HP Drain Rate
| 8
| 7
| 6
| 5
| 5
|}

= Results =

== Algorithmic Evaluation Results ==

120 beatmaps were checked using [https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/AiMod AiMod] across 4 models, 6 songs, and 5 difficulties.

'''Difficulty ordering warnings:'''
* DOSU - closer to you - Inner Oni < Oni
* MALer - YUBIKIRI-GENMAN - Inner Oni < Oni
* MALer - closer to you - Kantan < Oni < Futsuu < Inner Oni < Muzukashii

=== DOSU ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== FumenLab ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Objects <10ms apart and concurrent objects
| Objects <10ms apart and concurrent objects
|-
| A fight in Tunisia
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
|-
| yubikiri-genman
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== HiTaiko ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| OK
|}

=== MALer ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
|}

== Expert Evaluation Results ==

5 osu!taiko mappers evaluated submissions across 5 songs: '''Silver Bullet''', '''A fight in Tunisia''', '''YUBIKIRI-GENMAN''', '''Something Fun''', and '''closer to you'''. Each judge was assigned 3 songs based on their familiarity and interest of the songs, and each song was evaluated by 3 judges. For every assigned song, judges assessed submissions from all 4 models on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10.

We follow [https://osu.ppy.sh/wiki/en/Contests/Monthly_Beatmapping_Contest Monthly Beatmapping Contest] and adopt '''within-song normalization''' for per-song report. For each category, judge X's final score = (judge X's raw score - judge X's average raw score) / judge X's standard deviation within that song, and a submission's score is the sum of each judge's final score.

=== Silver Bullet ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| '''1.5872'''
| −2.6304
| '''2.0099'''
| −0.4552
| Musical representation drew mixed assessments. Oni and Muzukashii showed effective sound selection, but Inner Oni was conservative early and uneven near the ending. Inconsistent special-object use and variable-BPM handling weakened coherence across difficulties.
|-
| FumenLab
| 0.3005
| 0.9401
| −3.5863
| −1.9478
| Oni and Inner Oni showed promising sound selection, but awkward bursts and near-overlapping notes compromised gameplay near the ending. Kantan was excessively sparse, while intermediate difficulties were comparatively plain. Repeated musical phrases received inconsistent treatment.
|-
| HiTaiko
| 1.2778
| '''1.8629'''
| '''2.0099'''
| '''4.1634'''
| Coherent musical representation and expressive color patterns were recurring strengths. Inner Oni made quieter passages disproportionately dense, while Oni was comparatively restrained. Futsuu and Muzukashii showed promising structure, although assessments of the lower-tier ending differed. Finisher placement remained inconsistent.
|-
| MALer
| −3.1656
| −0.1726
| −0.4335
| −1.7604
| Imaginative Oni patterns and distinctive sectioning were offset by rhythms that often felt disconnected from the music. Inner Oni contained timing and overlap problems; unusual subdivisions in Muzukashii and questionable sound selection in Futsuu weakened the difficulty spread.
|}

=== A fight in Tunisia ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −4.7037
| −3.6090
| 0.4677
| −1.5881
| The mapping frequently missed prominent musical events across all difficulties. Conservative patterns supported playability, but omitted piano passages and notes in musically empty sections weakened representation. Isolated Inner Oni patterns showed potential without fitting the song.
|-
| FumenLab
| 0.2720
| −0.2284
| −2.4438
| −0.1690
| Awkward bursts, excessive local density and neglected piano passages weakened musical clarity. Kantan–Muzukashii lacked coherent sound selection; Oni was sometimes too demanding for its tier. Inner Oni was relatively stronger but remained rhythmically awkward.
|-
| HiTaiko
| 2.0747
| 0.9776
| '''2.2563'''
| '''2.3546'''
| Oni stood out for coherent sound selection and structure. Inner Oni was more conservative and relied on abrupt doublets, while Kantan–Muzukashii often omitted material or mixed musical layers inconsistently. Piano representation and repeated special-object placement remained weaknesses.
|-
| MALer
| '''2.3570'''
| '''2.8598'''
| −0.2803
| −0.5975
| Expressive color patterns attracted praise, but overall assessments differed. Complex snapping and inconsistent musical-layer selection reduced readability, particularly in Kantan–Muzukashii. Inner Oni offered some musical representation, although its structure and rhythmic emphasis remained uneven.
|}

=== YUBIKIRI-GENMAN ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −1.6143
| '''0.6706'''
| −1.2916
| −1.5202
| Excessive rhythmic improvisation disrupted the song’s gentle character. Muzukashii, Oni and Inner Oni often emphasized verses over choruses, weakening emotional progression. Kantan’s simpler rhythms were more acceptable, but consistency, object placement and musical correspondence remained concerns.
|-
| FumenLab
| 1.3093
| 0.3325
| '''2.5865'''
| 0.6172
| Assessments of musical coherence differed. Muzukashii and Oni sometimes followed the vocals effectively, but repeated phrases received inconsistent treatment. Inner Oni’s complex climax disrupted the gentle atmosphere. Kantan was sparse, and hitsound volume and effect choices required refinement.
|-
| HiTaiko
| '''1.3093'''
| −0.2970
| 1.8815
| '''4.0095'''
| Oni and Inner Oni showed effective vocal following and a convincing build toward the final chorus, although some feedback found them too conservative. Muzukashii’s density allocation was inconsistent; Kantan and Futsuu offered little content. Special-object use and omissions remained concerns.
|-
| MALer
| −1.0043
| −0.7060
| −3.1764
| −3.1065
| Irregular snapping and abrupt density changes were poorly suited to the gentle vocal/piano arrangement. Muzukashii, Oni and Inner Oni contained isolated difficulty spikes, with Oni reportedly harder than Inner Oni. Some color patterns and brief rhythm choices worked but were not sustained.
|}
''Musical Representation scores for FumenLab and HiTaiko round to the same value; HiTaiko has the higher unrounded score.''

=== Something Fun ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −4.1362
| −3.0094
| −2.8967
| −3.9851
| Rhythmic choices and sound selection were poorly aligned with the music. Missing notes, excessive finishers and misplaced spinners weakened consistency. Both Muzukashii and Inner Oni contained passages whose musical intent was unclear.
|-
| FumenLab
| 1.7808
| 2.0497
| '''2.0645'''
| '''2.1521'''
| The set was broadly serviceable, but repetitive color patterns and inconsistent finishers limited expressiveness. Kantan and Futsuu were weak, while some Inner Oni streams added unnecessary complexity. Isolated passages also raised timing concerns.
|-
| HiTaiko
| −1.0707
| −1.3013
| −0.3393
| 0.6620
| Some patterns followed the melody effectively, but prolonged simple rhythms, questionable drumrolls and uneven sectional difficulty made the experience repetitive. Finisher placement lacked consistency; no particular difficulty tier emerged as a clear strength.
|-
| MALer
| '''3.4261'''
| '''2.2610'''
| 1.1716
| 1.1710
| Creative ideas produced sharply divided assessments. Futsuu omitted a conspicuous musical passage, while Inner Oni showed uneven density and timing problems. Kantan and Muzukashii had questionable rhythmic choices; Oni was comparatively more successful but appeared too easy for its label.
|}

=== closer to you ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −0.5579
| 0.7683
| −0.7678
| −0.5907
| Rhythmic improvisation showed promise, and Futsuu conveyed a pleasing feel. However, inconsistent finishers and uneven density weakened progression: Oni omitted the opening, while Inner Oni used fewer streams than Muzukashii.
|-
| FumenLab
| '''3.3983'''
| '''4.2866'''
| '''4.4877'''
| '''3.5659'''
| Strong structure and effective rhythmic improvisation were recurring positives, with Inner Oni receiving particular praise. Color-pattern variety, consistency and timing-point use still needed refinement, but the set conveyed a comparatively convincing mapping approach.
|-
| HiTaiko
| 1.6737
| −2.5275
| −0.1399
| 1.0151
| Patterns were coherent but repetitive, with little distinction between difficulty tiers. Even Inner Oni remained sparse and closer to a much easier difficulty. Drumroll use and the ending finisher were questionable choices for the song’s calm character.
|-
| MALer
| −4.5141
| −2.5275
| −3.5800
| −3.9903
| Missing sections, timing errors and inconsistent musical logic undermined the spread. Muzukashii was reportedly the hardest tier, while easier difficulties contained unexplained breaks. Inner Oni’s awkward fast bursts did not resolve the overall lack of musical coverage.
|}

Qualitative feedback suggested distinct tendencies across the evaluated models. HiTaiko often favored coherent but conservative arrangements, while MALer explored more unusual rhythms with less consistent musical fit. FumenLab demonstrated effective improvisation and higher-difficulty patterning, but struggled with simplification and technical consistency. DOSU showed isolated strengths in sound selection, alongside substantial variation in musical correspondence and structural coherence. Across models, adapting musical ideas to lower difficulties remained a recurring challenge.

== Community Evaluation Results ==

{| class="wikitable"
|-
! Mapper
! Musical Representation
! Creativity
! Gameplay
! Overall Quality Score
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
|
|
|
|
|
|
|-
| FumenLab
|
|
|
|
|
|
|-
| HiTaiko
|
|
|
|
|
|
|-
| MALer
|
|
|
|
|
|
|-
| Amasugi (human mapper)
|
|
|
|
|
|
|}

2026:Rhythm Game Chart Generation Results

2026-10-14T08:32:31Z

Carolina Carusi: /* Community Evaluation Results */

= Submissions =

{| class="wikitable"
|-
! Submission
! Title
! PDF
! Authors
|-
| DOSU
| DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/DOSU.pdf PDF]
| Junyoung Koh, Hao-Wen Dong
|-
| FumenLab
| FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/FumenLab.pdf PDF]
| Ryo Mitoma
|-
| HiTaiko
| HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/HiTaiko.pdf PDF]
| Sungjung Park
|-
| MALer
| RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/MALer.pdf PDF]
| Sojeong An, Chris Donahue, Dasaem Jeong
|}

= Test Set =

=== Song Selection ===

{| class="wikitable"
|-
! Title
! Artist
! Genre
! Language
! Tags
! BPM
|-
| Silver Bullet
| Ennnn
| Electronic
| Instrumental
| breakcore, neurofunk, tech
| 89-379 (mostly 175)
|-
| A fight in Tunisia
| Sobrem & NIWASHI
| Jazz
| Instrumental
| piano
| 148
|-
| Something Fun
| JOYLESS
| Rock
| English
| male vocal
| 194
|-
| closer to you (than ever before)
| xiiiac13
| Electronic
| Instrumental
| lo-fi
| 110
|-
| YUBIKIRI-GENMAN
| Mili
| Pop
| Japanese
| j-pop, progressive, female vocal
| 59-146 (mostly 146)
|-
| Himitsu feat. KAFU
| MIMI
| Pop
| Japanese
| j-pop, vocaloid
| 168
|}

=== Default Settings ===

{| class="wikitable"
|-
! Parameter
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Overall Difficulty
| 3
| 4
| 5
| 5
| 6
|-
| HP Drain Rate
| 8
| 7
| 6
| 5
| 5
|}

= Results =

== Algorithmic Evaluation Results ==

120 beatmaps were checked using [https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/AiMod AiMod] across 4 models, 6 songs, and 5 difficulties.

'''Difficulty ordering warnings:'''
* DOSU - closer to you - Inner Oni < Oni
* MALer - YUBIKIRI-GENMAN - Inner Oni < Oni
* MALer - closer to you - Kantan < Oni < Futsuu < Inner Oni < Muzukashii

=== DOSU ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== FumenLab ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Objects <10ms apart and concurrent objects
| Objects <10ms apart and concurrent objects
|-
| A fight in Tunisia
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
|-
| yubikiri-genman
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== HiTaiko ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| OK
|}

=== MALer ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
|}

== Expert Evaluation Results ==

5 osu!taiko mappers evaluated submissions across 5 songs: '''Silver Bullet''', '''A fight in Tunisia''', '''YUBIKIRI-GENMAN''', '''Something Fun''', and '''closer to you'''. Each judge was assigned 3 songs based on their familiarity and interest of the songs, and each song was evaluated by 3 judges. For every assigned song, judges assessed submissions from all 4 models on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10.

We follow [https://osu.ppy.sh/wiki/en/Contests/Monthly_Beatmapping_Contest Monthly Beatmapping Contest] and adopt '''within-song normalization''' for per-song report. For each category, judge X's final score = (judge X's raw score - judge X's average raw score) / judge X's standard deviation within that song, and a submission's score is the sum of each judge's final score.

=== Silver Bullet ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| '''1.5872'''
| −2.6304
| '''2.0099'''
| −0.4552
| Musical representation drew mixed assessments. Oni and Muzukashii showed effective sound selection, but Inner Oni was conservative early and uneven near the ending. Inconsistent special-object use and variable-BPM handling weakened coherence across difficulties.
|-
| FumenLab
| 0.3005
| 0.9401
| −3.5863
| −1.9478
| Oni and Inner Oni showed promising sound selection, but awkward bursts and near-overlapping notes compromised gameplay near the ending. Kantan was excessively sparse, while intermediate difficulties were comparatively plain. Repeated musical phrases received inconsistent treatment.
|-
| HiTaiko
| 1.2778
| '''1.8629'''
| '''2.0099'''
| '''4.1634'''
| Coherent musical representation and expressive color patterns were recurring strengths. Inner Oni made quieter passages disproportionately dense, while Oni was comparatively restrained. Futsuu and Muzukashii showed promising structure, although assessments of the lower-tier ending differed. Finisher placement remained inconsistent.
|-
| MALer
| −3.1656
| −0.1726
| −0.4335
| −1.7604
| Imaginative Oni patterns and distinctive sectioning were offset by rhythms that often felt disconnected from the music. Inner Oni contained timing and overlap problems; unusual subdivisions in Muzukashii and questionable sound selection in Futsuu weakened the difficulty spread.
|}

=== A fight in Tunisia ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −4.7037
| −3.6090
| 0.4677
| −1.5881
| The mapping frequently missed prominent musical events across all difficulties. Conservative patterns supported playability, but omitted piano passages and notes in musically empty sections weakened representation. Isolated Inner Oni patterns showed potential without fitting the song.
|-
| FumenLab
| 0.2720
| −0.2284
| −2.4438
| −0.1690
| Awkward bursts, excessive local density and neglected piano passages weakened musical clarity. Kantan–Muzukashii lacked coherent sound selection; Oni was sometimes too demanding for its tier. Inner Oni was relatively stronger but remained rhythmically awkward.
|-
| HiTaiko
| 2.0747
| 0.9776
| '''2.2563'''
| '''2.3546'''
| Oni stood out for coherent sound selection and structure. Inner Oni was more conservative and relied on abrupt doublets, while Kantan–Muzukashii often omitted material or mixed musical layers inconsistently. Piano representation and repeated special-object placement remained weaknesses.
|-
| MALer
| '''2.3570'''
| '''2.8598'''
| −0.2803
| −0.5975
| Expressive color patterns attracted praise, but overall assessments differed. Complex snapping and inconsistent musical-layer selection reduced readability, particularly in Kantan–Muzukashii. Inner Oni offered some musical representation, although its structure and rhythmic emphasis remained uneven.
|}

=== YUBIKIRI-GENMAN ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −1.6143
| '''0.6706'''
| −1.2916
| −1.5202
| Excessive rhythmic improvisation disrupted the song’s gentle character. Muzukashii, Oni and Inner Oni often emphasized verses over choruses, weakening emotional progression. Kantan’s simpler rhythms were more acceptable, but consistency, object placement and musical correspondence remained concerns.
|-
| FumenLab
| 1.3093
| 0.3325
| '''2.5865'''
| 0.6172
| Assessments of musical coherence differed. Muzukashii and Oni sometimes followed the vocals effectively, but repeated phrases received inconsistent treatment. Inner Oni’s complex climax disrupted the gentle atmosphere. Kantan was sparse, and hitsound volume and effect choices required refinement.
|-
| HiTaiko
| '''1.3093'''
| −0.2970
| 1.8815
| '''4.0095'''
| Oni and Inner Oni showed effective vocal following and a convincing build toward the final chorus, although some feedback found them too conservative. Muzukashii’s density allocation was inconsistent; Kantan and Futsuu offered little content. Special-object use and omissions remained concerns.
|-
| MALer
| −1.0043
| −0.7060
| −3.1764
| −3.1065
| Irregular snapping and abrupt density changes were poorly suited to the gentle vocal/piano arrangement. Muzukashii, Oni and Inner Oni contained isolated difficulty spikes, with Oni reportedly harder than Inner Oni. Some color patterns and brief rhythm choices worked but were not sustained.
|}
''Musical Representation scores for FumenLab and HiTaiko round to the same value; HiTaiko has the higher unrounded score.''

=== Something Fun ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −4.1362
| −3.0094
| −2.8967
| −3.9851
| Rhythmic choices and sound selection were poorly aligned with the music. Missing notes, excessive finishers and misplaced spinners weakened consistency. Both Muzukashii and Inner Oni contained passages whose musical intent was unclear.
|-
| FumenLab
| 1.7808
| 2.0497
| '''2.0645'''
| '''2.1521'''
| The set was broadly serviceable, but repetitive color patterns and inconsistent finishers limited expressiveness. Kantan and Futsuu were weak, while some Inner Oni streams added unnecessary complexity. Isolated passages also raised timing concerns.
|-
| HiTaiko
| −1.0707
| −1.3013
| −0.3393
| 0.6620
| Some patterns followed the melody effectively, but prolonged simple rhythms, questionable drumrolls and uneven sectional difficulty made the experience repetitive. Finisher placement lacked consistency; no particular difficulty tier emerged as a clear strength.
|-
| MALer
| '''3.4261'''
| '''2.2610'''
| 1.1716
| 1.1710
| Creative ideas produced sharply divided assessments. Futsuu omitted a conspicuous musical passage, while Inner Oni showed uneven density and timing problems. Kantan and Muzukashii had questionable rhythmic choices; Oni was comparatively more successful but appeared too easy for its label.
|}

=== closer to you ===

{| class="wikitable"
! Model
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Qualitative Feedback
|-
| DOSU
| −0.5579
| 0.7683
| −0.7678
| −0.5907
| Rhythmic improvisation showed promise, and Futsuu conveyed a pleasing feel. However, inconsistent finishers and uneven density weakened progression: Oni omitted the opening, while Inner Oni used fewer streams than Muzukashii.
|-
| FumenLab
| '''3.3983'''
| '''4.2866'''
| '''4.4877'''
| '''3.5659'''
| Strong structure and effective rhythmic improvisation were recurring positives, with Inner Oni receiving particular praise. Color-pattern variety, consistency and timing-point use still needed refinement, but the set conveyed a comparatively convincing mapping approach.
|-
| HiTaiko
| 1.6737
| −2.5275
| −0.1399
| 1.0151
| Patterns were coherent but repetitive, with little distinction between difficulty tiers. Even Inner Oni remained sparse and closer to a much easier difficulty. Drumroll use and the ending finisher were questionable choices for the song’s calm character.
|-
| MALer
| −4.5141
| −2.5275
| −3.5800
| −3.9903
| Missing sections, timing errors and inconsistent musical logic undermined the spread. Muzukashii was reportedly the hardest tier, while easier difficulties contained unexplained breaks. Inner Oni’s awkward fast bursts did not resolve the overall lack of musical coverage.
|}

Qualitative feedback suggested distinct tendencies across the evaluated models. HiTaiko often favored coherent but conservative arrangements, while MALer explored more unusual rhythms with less consistent musical fit. FumenLab demonstrated effective improvisation and higher-difficulty patterning, but struggled with simplification and technical consistency. DOSU showed isolated strengths in sound selection, alongside substantial variation in musical correspondence and structural coherence. Across models, adapting musical ideas to lower difficulties remained a recurring challenge.

== Community Evaluation Results ==

{| class="wikitable"
|-
! Mapper
! Musical Representation
! Creativity
! Gameplay
! Overall Quality Score
! Human-Likeness
|-
| DOSU
|
|
|
|
|
|-
| FumenLab
|
|
|
|
|
|-
| HiTaiko
|
|
|
|
|
|-
| MALer
|
|
|
|
|
|-
| Amasugi (human mapper)
|
|
|
|
|
|}

2026:Rhythm Game Chart Generation Results

2026-10-11T09:09:56Z

Carolina Carusi:

= Submissions =

{| class="wikitable"
|-
! Submission
! Title
! PDF
! Authors
|-
| DOSU
| DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION
|
| Junyoung Koh, Hao-Wen Dong
|-
| FumenLab
| FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS
|
| Ryo Mitoma
|-
| HiTaiko
| HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection
|
| Sungjung Park
|-
| MALer
| RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL
|
| Sojeong An, Chris Donahue, Dasaem Jeong
|}

= Test Set =

=== Song Selection ===

{| class="wikitable"
|-
! Title
! Artist
! Genre
! Language
! Tags
! BPM
|-
| Silver Bullet
| Ennnn
| Electronic
| Instrumental
| breakcore, neurofunk, tech
| 89-379 (mostly 175)
|-
| A fight in Tunisia
| Sobrem & NIWASHI
| Jazz
| Instrumental
| piano
| 148
|-
| Something Fun
| JOYLESS
| Rock
| English
| male vocal
| 194
|-
| closer to you (than ever before)
| xiiiac13
| Electronic
| Instrumental
| lo-fi
| 110
|-
| YUBIKIRI-GENMAN
| Mili
| Pop
| Japanese
| j-pop, progressive, female vocal
| 59-146 (mostly 146)
|-
| Himitsu feat. KAFU
| MIMI
| Pop
| Japanese
| j-pop, vocaloid
| 168
|}

=== Default Settings ===

{| class="wikitable"
|-
! Parameter
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Overall Difficulty
| 3
| 4
| 5
| 5
| 6
|-
| HP Drain Rate
| 8
| 7
| 6
| 5
| 5
|}

= Results =

== Algorithmic Evaluation Results ==

120 beatmaps were checked using [https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/AiMod AiMod] across 4 models, 6 songs, and 5 difficulties.

Difficulty ordering warnings:
* DOSU - closer to you - Inner Oni < Oni
* MALer - YUBIKIRI-GENMAN - Inner Oni < Oni
* MALer - closer to you - Kantan < Oni < Futsuu < Inner Oni < Muzukashii

=== DOSU ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== FumenLab ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Objects <10ms apart and concurrent objects
| Objects <10ms apart and concurrent objects
|-
| A fight in Tunisia
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
|-
| yubikiri-genman
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== HiTaiko ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| OK
|}

=== MALer ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
|}

== Expert Evaluation Results ==

== Community Evaluation Results ==

{| class="wikitable"
|-
! Mapper
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Overall Mean
|-
| DOSU
|
|
|
|
|
|-
| FumenLab
|
|
|
|
|
|-
| MALer
|
|
|
|
|
|-
| HiTaiko
|
|
|
|
|
|-
| Amasugi (human mapper)
|
|
|
|
|
|}

2026:Rhythm Game Chart Generation Results

2026-10-11T09:09:39Z

Carolina Carusi:

= Submissions =

{| class="wikitable"
|-
! Submission
! Title
! PDF
! Authors
|-
| DOSU
| DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION
|
| Junyoung Koh, Hao-Wen Dong
|-
| FumenLab
| FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS
|
| Ryo Mitoma
|-
| HiTaiko
| HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection
|
| Sungjung Park
|-
| MALer
| RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL
|
| Sojeong An, Chris Donahue, Dasaem Jeong
|}

= Test Set =

=== Song Selection ===

{| class="wikitable"
|-
! Title
! Artist
! Genre
! Language
! Tags
! BPM
|-
| Silver Bullet
| Ennnn
| Electronic
| Instrumental
| breakcore, neurofunk, tech
| 89-379 (mostly 175)
|-
| A fight in Tunisia
| Sobrem & NIWASHI
| Jazz
| Instrumental
| piano
| 148
|-
| Something Fun
| JOYLESS
| Rock
| English
| male vocal
| 194
|-
| closer to you (than ever before)
| xiiiac13
| Electronic
| Instrumental
| lo-fi
| 110
|-
| YUBIKIRI-GENMAN
| Mili
| Pop
| Japanese
| j-pop, progressive, female vocal
| 59-146 (mostly 146)
|-
| Himitsu feat. KAFU
| MIMI
| Pop
| Japanese
| j-pop, vocaloid
| 168
|}

=== Default Settings ===

{| class="wikitable"
|-
! Parameter
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Overall Difficulty
| 3
| 4
| 5
| 5
| 6
|-
| HP Drain Rate
| 8
| 7
| 6
| 5
| 5
|}

= Results =

== Algorithmic Evaluation Results ==

120 beatmaps were checked using [https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/AiMod AiMod] across 4 models, 6 songs, and 5 difficulties.

Difficulty ordering warnings:
* DOSU - closer to you - Inner Oni < Oni
* MALer - YUBIKIRI-GENMAN - Inner Oni < Oni
* MALer - closer to you - Kantan < Oni < Futsuu < Inner Oni < Muzukashii

=== DOSU ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== FumenLab ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Objects <10ms apart and concurrent objects
| Objects <10ms apart and concurrent objects
|-
| A fight in Tunisia
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
|-
| yubikiri-genman
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== HiTaiko ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| OK
|}

=== MALer ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
|}

== Expert Evaluation Results ==

== Community Evaluation Results ==

{| class="wikitable"
|-
! Mapper
! Musical Representation
! Creativity
! Gameplay
! Human-Likeness
! Overall Mean
|-
| DOSU
|
|
|
|
|
|-
| FumenLab
|
|
|
|
|
|-
| MALer
|
|
|
|
|
|-
| HiTaiko
|
|
|
|
|
|-
| Amasugi (human mapper)
|
|
|
|
|
|}

test

2026:Rhythm Game Chart Generation Results

2026-10-11T08:37:55Z

Carolina Carusi: Algorithmic evaluation draft

= Submissions =

{| class="wikitable"
|-
! Submission
! Title
! PDF
! Authors
|-
| DOSU
| DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION
|
| Junyoung Koh, Hao-Wen Dong
|-
| FumenLab
| FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS
|
| Ryo Mitoma
|-
| HiTaiko
| HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection
|
| Sungjung Park
|-
| MALer
| RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL
|
| Sojeong An, Chris Donahue, Dasaem Jeong
|}

= Test Set =

=== Song Selection ===

{| class="wikitable"
|-
! Title
! Artist
! Genre
! Language
! Tags
! BPM
|-
| Silver Bullet
| Ennnn
| Electronic
| Instrumental
| breakcore, neurofunk, tech
| 89-379 (mostly 175)
|-
| A fight in Tunisia
| Sobrem & NIWASHI
| Jazz
| Instrumental
| piano
| 148
|-
| Something Fun
| JOYLESS
| Rock
| English
| male vocal
| 194
|-
| closer to you (than ever before)
| xiiiac13
| Electronic
| Instrumental
| lo-fi
| 110
|-
| YUBIKIRI-GENMAN
| Mili
| Pop
| Japanese
| j-pop, progressive, female vocal
| 59-146 (mostly 146)
|-
| Himitsu feat. KAFU
| MIMI
| Pop
| Japanese
| j-pop, vocaloid
| 168
|}

=== Default Settings ===

= Results =

== Algorithmic Evaluation Results ==

120 beatmaps were checked across 4 models, 6 songs, and 5 difficulties.
Difficulty ordering warnings:
* DOSU - closer to you - Inner Oni < Oni
* MALer - YUBIKIRI-GENMAN - Inner Oni < Oni
* MALer - closer to you - Kantan < Oni < Futsuu < Inner Oni < Muzukashii

=== DOSU ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== FumenLab ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Objects <10ms apart and concurrent objects
| Objects <10ms apart and concurrent objects
|-
| A fight in Tunisia
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
|-
| yubikiri-genman
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| OK
| OK
| OK
| OK
| OK
|}

=== HiTaiko ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| Minor snapping issue on drumrolls and swells
| Minor snapping issue on drumrolls and swells
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| OK
|}

=== MALer ===

{| class="wikitable"
|-
! Song
! Kantan
! Futsuu
! Muzukashii
! Oni
! Inner Oni
|-
| Silver Bullet
| OK
| OK
| OK
| OK
| OK
|-
| A fight in Tunisia
| OK
| OK
| OK
| OK
| OK
|-
| Himitsu
| OK
| OK
| OK
| OK
| OK
|-
| yubikiri-genman
| OK
| OK
| OK
| OK
| OK
|-
| something fun
| OK
| OK
| OK
| OK
| OK
|-
| closer to you
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
| Unused audio at the end
|}

== Expert Evaluation Results ==

== Community Evaluation Results ==

2026:Rhythm Game Chart Generation

2026-09-10T08:00:20Z

Carolina Carusi: /* Timeline */

= Task Description =

Rhythm game is a genre of video games that require players to '''physically react in sync with music'''. A rhythm game chart consists of '''a sequence of timed note events synchronized with an audio track''', defining what actions players should perform and when players should perform these actions during gameplay. Rhythm game charting is a time-consuming task that requires musical expertise, and it is a common case where players want to play a song that does not have an existing chart. While increasing number of work has been done on automatic rhythm game chart generation in recent years, generating '''high-quality''' rhythm game charts that people enjoy playing remains a challenge.

To define the task within the scope of MIREX challenge, we specifically choose the task of generating '''osu!taiko''' beatmaps this year. osu!taiko is a game mode in osu! inspired by Taiko no Tatsujin, which features a horizontal conveyor belt of drum notes moving towards a judgment circle. In this game mode, players typically use a keyboard to strike different types of notes with precise timing. A high-quality mapping in osu!taiko involves many different aspects of music, including rhythmic perception, musical structure awareness, pitch awareness, musical instrument awareness, and so on.

[[File:hit_circle.jpg|300px]]
[[File:drumroll.jpg|300px]]
[[File:spinner.jpg|600px]]

In osu!, a beatmapset is a '''packed archive''' with <code>.osz</code> extension that contains a '''song''' in <code>MP3</code> or <code>Ogg</code> format, a set of '''game levels (difficulties)''', and other audio and visual materials. Each game level is stored in a '''text file''' with <code>.osu</code> extension, which includes 7 sections: General, Editor, Metadata, Difficulty, Events, Timing Points, and Hit Objects. This year, we specify the task as predicting '''hit objects (mandatory)''' and '''inherited timing points (optional)''' based on the '''audio''', '''difficulty level''', and '''ground truth uninherited timing points'''. While existing research mostly focuses on hit objects, inherited timing points are also critical factors to carry musical meanings and make a chart feel human-like. Consequently, we include the generation of inherited timing points as an optional task to encourage participants to push the boundaries of current research. To ensure consistency, we restrict the difficulty levels to be Kantan, Futsuu, Muzukashii, Oni, and Inner Oni (Ura Oni).

[[File:file_tree.png|400px]]
[[File:osu_file.png|400px]]

= Evaluation Criteria =

While existing research relies on researcher-defined quantitative evaluation methods, we adopt a more user-centered 3-stage approach. The following evaluation will be performed on songs that are not included in the dataset provided to the participants. We will recruit about 5 osu!taiko mappers as the jury, and the jury will also be involved in the test song selection process.

=== Algorithmic Evaluation ===

In this stage, we will manually pick at least 3 songs for evaluation. First, we will use [https://pypi.org/project/osu-sr-calculator/ osu! star rating calculator] to calculate the star ratings of the generated beatmaps. This allows us to assess whether the model's difficulty conditioning is functioning correctly. Then, we will follow [https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/AiMod AiMod] and [https://github.com/Naxesss/MapsetVerifier MapsetVerifier] to detect unplayable placements of objects algorithmically. Models passing this evaluation phase will move on to human evaluation.

=== Expert Evaluation ===

In this stage, the jury will rate the generated charts based on musical representation, creativity, and gameplay similar to the [https://osu.ppy.sh/wiki/en/Contests/Monthly_Beatmapping_Contest Monthly Beatmapping Contest], and we will also introduce a humanity score evaluating how human-like the generated charts are. We will then use the jury's ratings to determine which finalists move on to player evaluation.

=== Community Evaluation ===

In this stage, a single showcase song will be selected for community evaluation. Participating players will choose a difficulty level aligned with their skill level and playtest charts generated by each model alongside a human-authored chart. Players will vote on musical representation, creativity, gameplay, and human-likeness of the generated charts.

= Resources =
=== General ===

We recommend participants to read [https://docs.google.com/document/d/1WyoOXPpwVaLPXJeP-4SrGO6ywxEbbkKwVbg8jDfgtnI/edit?tab=t.0 this document] to get an overall idea of how osu!taiko mapping works from a mapper's perspective. This is especially helpful if you are not familiar with osu!taiko gaming and/or mapping.

=== osu! wiki ===

Participants are encouraged to read osu! wiki to gain a clearer understanding of the gameplay, data format, timing, and mapping goals. We hereby list some very useful pages:

* osu!taiko: https://osu.ppy.sh/wiki/en/Game_mode/osu%21taiko
* .osu file format: https://osu.ppy.sh/wiki/en/Client/File_formats/osu_%28file_format%29
* hitsound: https://osu.ppy.sh/wiki/en/Beatmapping/Hitsound
* timing: https://osu.ppy.sh/wiki/en/Beatmapping/Timing
* timing tab: https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/Timing
* ranking criteria: https://osu.ppy.sh/wiki/en/Ranking_criteria
* osu!taiko ranking criteria: https://osu.ppy.sh/wiki/en/Ranking_criteria/osu%21taiko
* beat snap divisor: https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/Beat_snap_divisor

=== Community GitHub repositories ===

List of popular GitHub repositories for osu! beatmap generation models without papers:

* Mapperatorinator: https://github.com/OliBomby/Mapperatorinator
* osu!dreamer: https://github.com/jaswon/osu-dreamer
* Beat Learning: https://github.com/sedthh/BeatLearning
* osumapper: https://github.com/kotritrona/osumapper

=== Papers and extended abstracts ===

List of papers and extended abstracts on osu! beatmap generation:

* Hanzen, Jonah, Emily Halina, and Matthew Guzdial. "Time-based Chart Partitioning: Improving Local Coherency in Rhythm Game Chart Generation." ''Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment''. Vol. 21. No. 1. 2025.
* Yi, Jayeon, Sungho Lee, and Kyogu Lee. "Beat-Aligned Spectrogram-to-Sequence Generation of Rhythm-Game Charts." ''ISMIR LBD''. 2023. GitHub repo: https://github.com/stet-stet/goct_ismir2023
* Lee, Sihun, and Dasaem Jeong. "AutoOsu: Audio-Aware Action Generation for Rhythm Games." ''ISMIR LBD''. 2023. GitHub repo: https://github.com/issyun/AutoOsu
* Halina, Emily, and Matthew Guzdial. "TaikoNation: Patterning-focused chart generation for rhythm action games." ''Proceedings of the 16th International Conference on the Foundations of Digital Games''. 2021. GitHub repo: https://github.com/emily-halina/TaikoNationV1

= Training Dataset =

To ensure data quality and fairness, we require all participants to use beatmaps ranked from '''May 5, 2019''' to '''December 31, 2025''' for training and validation. We recommend using [https://osudl.org/ this website] and filter for '''"ranked>=2019-05-05 and ranked<2026-01-01 and mode=t"''', which yields '''727''' beatmapsets. To minimize legal risks and save resources, we also encourage participants to alternatively filter for '''"ranked>=2019-05-05 and ranked<2026-01-01 and mode=t and featured artist"''', which yields '''174''' beatmapsets. For more information on featured artists, see [https://osu.ppy.sh/beatmaps/artists this page].

In osu!taiko, there are usually Kantan, Futsuu, Muzukashii, Oni, and Inner Oni difficulty levels. However, sometimes people may use different names for them, and there also exist levels that are higher than Inner Oni. To ensure consistency, we recommend using the following tier classification logic:

<pre>
function get_tier(osu_file):

raw_name = osu_file.version_name.lower()

# Keep only a-z and '
clean_name = RegexReplace(raw_name, "[^a-z']", "")

if "kantan" in clean_name:
return 1

elif "futsuu" in clean_name:
return 2

elif "muzukashii" in clean_name:
return 3

elif "oni" in clean_name:
if "inner" in clean_name or "ura" in clean_name:
return 5

elif "hell" in clean_name:
return 6

elif clean_name == "oni" or clean_name matches "^.+'soni$":
return 4

return None # Discard non-standard variants
</pre>

= Submission Requirements =

Participants are required to submit the source code and model checkpoints of their systems via a link to a '''Git repository (specifying a unique commit hash)''' or a '''Docker image digest''' by '''September 11, 2026'''. Participants are free to convert the raw data into any internal format that best suits their model architecture, provided the complete pipeline is included in the submission. Submissions must be accompanied by a '''technical report in PDF format (2-4 pages)''' detailing the '''model architecture, dataset, and training process'''.

Following the system submission deadline, the '''hidden test set''' will be released. This set will include at least 3 '''template <code>.osz</code> files''', each containing an '''audio file''' and '''5 template <code>.osu</code> files''' with ground truth uninherited timing points. Participants will have a '''>= 48-hour window''' to execute their ''frozen'' systems on their own hardware of choice to generate the required outputs. For each song in the test set, the system should generate '''hit objects (mandatory)''' and '''inherited timing points (optional)''' for '''all 5 difficulty levels''' (Kantan, Futsuu, Muzukashii, Oni, Inner Oni). By the end of the >= 48-hour window, participants should submit their populated <code>.osz</code> files containing the completed <code>.osu</code> files. Please include your '''contact information''' in your technical report so that we can send the templates to you directly.

Note that some attributes (such as x and y axes) in hit objects are not relevant to osu!taiko gameplay, and therefore should not be predicted (see [https://osu.ppy.sh/wiki/en/Client/File_formats/osu_%28file_format%29 this page] for details). In the submission, participants can set these attributes to default values (e.g. 256 for x and 192 for y).

Also note that for the optional prediction of inherited timing points, participants should standardize velocity multiplier calculation to account for the relationship between the global multiplier in the Difficulty section and local multipliers in inherited timing points. Template <code>.osu</code> files usually have a global multiplier of 1.4.

The organizers reserve the right to verify the integrity of any submission by performing a local execution of the provided Docker digest or Git commit hash to ensure bit-perfect or functionally equivalent reproduction of the submitted maps.

= Timeline =

* '''July 1, 2026''': Submission system opens
* '''Sep 11, 2026''': Submission deadline for the generative system (Git repository/Docker image) + report (PDF, 2-4 pages)
* '''Sep 12, 2026''': Test set released
* '''Sep 14, 2026''': Submission deadline for the generated beatmaps (populated <code>.osz</code> files containing <code>.osu</code> files)
* '''Oct 15, 2026''': Results published

All the deadlines above are before '''midnight, Anywhere on Earth'''.

= Organizers =

* Ziyun Liu — [mailto:ziyun.liu@univ-lille.fr ziyun.liu@univ-lille.fr]
* Carolina Carusi — [mailto:carolina.carusi@kaist.ac.kr carolina.carusi@kaist.ac.kr]

= Ethical Statement =

We are fully aware of the negative impact generative AI systems can have on the osu! community and strongly oppose their deployment beyond research or personal use. Participants are strictly prohibited from distributing trained models publicly, advertising them, or offering public-facing user interfaces. Following direct communication with mappers, we have identified key community needs that will shape future iterations of this task.

2026:Rhythm Game Chart Generation

2026-09-10T07:59:24Z

Carolina Carusi:

= Task Description =

Rhythm game is a genre of video games that require players to '''physically react in sync with music'''. A rhythm game chart consists of '''a sequence of timed note events synchronized with an audio track''', defining what actions players should perform and when players should perform these actions during gameplay. Rhythm game charting is a time-consuming task that requires musical expertise, and it is a common case where players want to play a song that does not have an existing chart. While increasing number of work has been done on automatic rhythm game chart generation in recent years, generating '''high-quality''' rhythm game charts that people enjoy playing remains a challenge.

To define the task within the scope of MIREX challenge, we specifically choose the task of generating '''osu!taiko''' beatmaps this year. osu!taiko is a game mode in osu! inspired by Taiko no Tatsujin, which features a horizontal conveyor belt of drum notes moving towards a judgment circle. In this game mode, players typically use a keyboard to strike different types of notes with precise timing. A high-quality mapping in osu!taiko involves many different aspects of music, including rhythmic perception, musical structure awareness, pitch awareness, musical instrument awareness, and so on.

[[File:hit_circle.jpg|300px]]
[[File:drumroll.jpg|300px]]
[[File:spinner.jpg|600px]]

In osu!, a beatmapset is a '''packed archive''' with <code>.osz</code> extension that contains a '''song''' in <code>MP3</code> or <code>Ogg</code> format, a set of '''game levels (difficulties)''', and other audio and visual materials. Each game level is stored in a '''text file''' with <code>.osu</code> extension, which includes 7 sections: General, Editor, Metadata, Difficulty, Events, Timing Points, and Hit Objects. This year, we specify the task as predicting '''hit objects (mandatory)''' and '''inherited timing points (optional)''' based on the '''audio''', '''difficulty level''', and '''ground truth uninherited timing points'''. While existing research mostly focuses on hit objects, inherited timing points are also critical factors to carry musical meanings and make a chart feel human-like. Consequently, we include the generation of inherited timing points as an optional task to encourage participants to push the boundaries of current research. To ensure consistency, we restrict the difficulty levels to be Kantan, Futsuu, Muzukashii, Oni, and Inner Oni (Ura Oni).

[[File:file_tree.png|400px]]
[[File:osu_file.png|400px]]

= Evaluation Criteria =

While existing research relies on researcher-defined quantitative evaluation methods, we adopt a more user-centered 3-stage approach. The following evaluation will be performed on songs that are not included in the dataset provided to the participants. We will recruit about 5 osu!taiko mappers as the jury, and the jury will also be involved in the test song selection process.

=== Algorithmic Evaluation ===

In this stage, we will manually pick at least 3 songs for evaluation. First, we will use [https://pypi.org/project/osu-sr-calculator/ osu! star rating calculator] to calculate the star ratings of the generated beatmaps. This allows us to assess whether the model's difficulty conditioning is functioning correctly. Then, we will follow [https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/AiMod AiMod] and [https://github.com/Naxesss/MapsetVerifier MapsetVerifier] to detect unplayable placements of objects algorithmically. Models passing this evaluation phase will move on to human evaluation.

=== Expert Evaluation ===

In this stage, the jury will rate the generated charts based on musical representation, creativity, and gameplay similar to the [https://osu.ppy.sh/wiki/en/Contests/Monthly_Beatmapping_Contest Monthly Beatmapping Contest], and we will also introduce a humanity score evaluating how human-like the generated charts are. We will then use the jury's ratings to determine which finalists move on to player evaluation.

=== Community Evaluation ===

In this stage, a single showcase song will be selected for community evaluation. Participating players will choose a difficulty level aligned with their skill level and playtest charts generated by each model alongside a human-authored chart. Players will vote on musical representation, creativity, gameplay, and human-likeness of the generated charts.

= Resources =
=== General ===

We recommend participants to read [https://docs.google.com/document/d/1WyoOXPpwVaLPXJeP-4SrGO6ywxEbbkKwVbg8jDfgtnI/edit?tab=t.0 this document] to get an overall idea of how osu!taiko mapping works from a mapper's perspective. This is especially helpful if you are not familiar with osu!taiko gaming and/or mapping.

=== osu! wiki ===

Participants are encouraged to read osu! wiki to gain a clearer understanding of the gameplay, data format, timing, and mapping goals. We hereby list some very useful pages:

* osu!taiko: https://osu.ppy.sh/wiki/en/Game_mode/osu%21taiko
* .osu file format: https://osu.ppy.sh/wiki/en/Client/File_formats/osu_%28file_format%29
* hitsound: https://osu.ppy.sh/wiki/en/Beatmapping/Hitsound
* timing: https://osu.ppy.sh/wiki/en/Beatmapping/Timing
* timing tab: https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/Timing
* ranking criteria: https://osu.ppy.sh/wiki/en/Ranking_criteria
* osu!taiko ranking criteria: https://osu.ppy.sh/wiki/en/Ranking_criteria/osu%21taiko
* beat snap divisor: https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/Beat_snap_divisor

=== Community GitHub repositories ===

List of popular GitHub repositories for osu! beatmap generation models without papers:

* Mapperatorinator: https://github.com/OliBomby/Mapperatorinator
* osu!dreamer: https://github.com/jaswon/osu-dreamer
* Beat Learning: https://github.com/sedthh/BeatLearning
* osumapper: https://github.com/kotritrona/osumapper

=== Papers and extended abstracts ===

List of papers and extended abstracts on osu! beatmap generation:

* Hanzen, Jonah, Emily Halina, and Matthew Guzdial. "Time-based Chart Partitioning: Improving Local Coherency in Rhythm Game Chart Generation." ''Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment''. Vol. 21. No. 1. 2025.
* Yi, Jayeon, Sungho Lee, and Kyogu Lee. "Beat-Aligned Spectrogram-to-Sequence Generation of Rhythm-Game Charts." ''ISMIR LBD''. 2023. GitHub repo: https://github.com/stet-stet/goct_ismir2023
* Lee, Sihun, and Dasaem Jeong. "AutoOsu: Audio-Aware Action Generation for Rhythm Games." ''ISMIR LBD''. 2023. GitHub repo: https://github.com/issyun/AutoOsu
* Halina, Emily, and Matthew Guzdial. "TaikoNation: Patterning-focused chart generation for rhythm action games." ''Proceedings of the 16th International Conference on the Foundations of Digital Games''. 2021. GitHub repo: https://github.com/emily-halina/TaikoNationV1

= Training Dataset =

To ensure data quality and fairness, we require all participants to use beatmaps ranked from '''May 5, 2019''' to '''December 31, 2025''' for training and validation. We recommend using [https://osudl.org/ this website] and filter for '''"ranked>=2019-05-05 and ranked<2026-01-01 and mode=t"''', which yields '''727''' beatmapsets. To minimize legal risks and save resources, we also encourage participants to alternatively filter for '''"ranked>=2019-05-05 and ranked<2026-01-01 and mode=t and featured artist"''', which yields '''174''' beatmapsets. For more information on featured artists, see [https://osu.ppy.sh/beatmaps/artists this page].

In osu!taiko, there are usually Kantan, Futsuu, Muzukashii, Oni, and Inner Oni difficulty levels. However, sometimes people may use different names for them, and there also exist levels that are higher than Inner Oni. To ensure consistency, we recommend using the following tier classification logic:

<pre>
function get_tier(osu_file):

raw_name = osu_file.version_name.lower()

# Keep only a-z and '
clean_name = RegexReplace(raw_name, "[^a-z']", "")

if "kantan" in clean_name:
return 1

elif "futsuu" in clean_name:
return 2

elif "muzukashii" in clean_name:
return 3

elif "oni" in clean_name:
if "inner" in clean_name or "ura" in clean_name:
return 5

elif "hell" in clean_name:
return 6

elif clean_name == "oni" or clean_name matches "^.+'soni$":
return 4

return None # Discard non-standard variants
</pre>

= Submission Requirements =

Participants are required to submit the source code and model checkpoints of their systems via a link to a '''Git repository (specifying a unique commit hash)''' or a '''Docker image digest''' by '''September 11, 2026'''. Participants are free to convert the raw data into any internal format that best suits their model architecture, provided the complete pipeline is included in the submission. Submissions must be accompanied by a '''technical report in PDF format (2-4 pages)''' detailing the '''model architecture, dataset, and training process'''.

Following the system submission deadline, the '''hidden test set''' will be released. This set will include at least 3 '''template <code>.osz</code> files''', each containing an '''audio file''' and '''5 template <code>.osu</code> files''' with ground truth uninherited timing points. Participants will have a '''>= 48-hour window''' to execute their ''frozen'' systems on their own hardware of choice to generate the required outputs. For each song in the test set, the system should generate '''hit objects (mandatory)''' and '''inherited timing points (optional)''' for '''all 5 difficulty levels''' (Kantan, Futsuu, Muzukashii, Oni, Inner Oni). By the end of the >= 48-hour window, participants should submit their populated <code>.osz</code> files containing the completed <code>.osu</code> files. Please include your '''contact information''' in your technical report so that we can send the templates to you directly.

Note that some attributes (such as x and y axes) in hit objects are not relevant to osu!taiko gameplay, and therefore should not be predicted (see [https://osu.ppy.sh/wiki/en/Client/File_formats/osu_%28file_format%29 this page] for details). In the submission, participants can set these attributes to default values (e.g. 256 for x and 192 for y).

Also note that for the optional prediction of inherited timing points, participants should standardize velocity multiplier calculation to account for the relationship between the global multiplier in the Difficulty section and local multipliers in inherited timing points. Template <code>.osu</code> files usually have a global multiplier of 1.4.

The organizers reserve the right to verify the integrity of any submission by performing a local execution of the provided Docker digest or Git commit hash to ensure bit-perfect or functionally equivalent reproduction of the submitted maps.

= Timeline =

* '''July 1, 2026''': Submission system opens
* '''Sep 11, 2026 (midnight AoE)''': Submission deadline for the generative system (Git repository/Docker image) + report (PDF, 2-4 pages)
* '''Sep 12, 2026''': Test set released
* '''Sep 14, 2026 (midnight AoE)''': Submission deadline for the generated beatmaps (populated <code>.osz</code> files containing <code>.osu</code> files)
* '''Oct 15, 2026''': Results published

All the deadlines above are before midnight, Anywhere on Earth.

= Organizers =

* Ziyun Liu — [mailto:ziyun.liu@univ-lille.fr ziyun.liu@univ-lille.fr]
* Carolina Carusi — [mailto:carolina.carusi@kaist.ac.kr carolina.carusi@kaist.ac.kr]

= Ethical Statement =

We are fully aware of the negative impact generative AI systems can have on the osu! community and strongly oppose their deployment beyond research or personal use. Participants are strictly prohibited from distributing trained models publicly, advertising them, or offering public-facing user interfaces. Following direct communication with mappers, we have identified key community needs that will shape future iterations of this task.