Difference between revisions of "2026:Rhythm Game Chart Generation Results"

From MIREX Wiki
(→‎Test Set)
 
(39 intermediate revisions by 2 users not shown)
Line 10: Line 10:
 
| DOSU
 
| DOSU
 
| DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION
 
| DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION
−
|  
+
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/DOSU.pdf PDF]
 
| Junyoung Koh, Hao-Wen Dong
 
| Junyoung Koh, Hao-Wen Dong
 
|-
 
|-
 
| FumenLab
 
| FumenLab
 
| FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS
 
| FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS
−
|  
+
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/FumenLab.pdf PDF]
 
| Ryo Mitoma
 
| Ryo Mitoma
 
|-
 
|-
 
| HiTaiko
 
| HiTaiko
 
| HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection
 
| HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection
−
|  
+
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/HiTaiko.pdf PDF]
 
| Sungjung Park
 
| Sungjung Park
 
|-
 
|-
 
| MALer
 
| MALer
 
| RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL
 
| RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL
−
|  
+
| [http://futuremirex.com/portal/wp-content/uploads/2026/rhythm-game-chart-generation/MALer.pdf PDF]
 
| Sojeong An, Chris Donahue, Dasaem Jeong
 
| Sojeong An, Chris Donahue, Dasaem Jeong
 
|}
 
|}
  
 
= Test Set =
 
= Test Set =
 +
 +
=== Song Selection ===
  
 
{| class="wikitable"  
 
{| class="wikitable"  
Line 38: Line 40:
 
! Language
 
! Language
 
! Tags
 
! Tags
 +
! BPM
 
|-
 
|-
 
| Silver Bullet
 
| Silver Bullet
Line 44: Line 47:
 
| Instrumental
 
| Instrumental
 
| breakcore, neurofunk, tech
 
| breakcore, neurofunk, tech
 +
| 89-379 (mostly 175)
 
|-
 
|-
 
| A fight in Tunisia
 
| A fight in Tunisia
Line 50: Line 54:
 
| Instrumental
 
| Instrumental
 
| piano
 
| piano
 +
| 148
 
|-
 
|-
 
| Something Fun
 
| Something Fun
Line 56: Line 61:
 
| English
 
| English
 
| male vocal
 
| male vocal
 +
| 194
 
|-
 
|-
 
| closer to you (than ever before)
 
| closer to you (than ever before)
Line 62: Line 68:
 
| Instrumental
 
| Instrumental
 
| lo-fi
 
| lo-fi
 +
| 110
 
|-
 
|-
 
| YUBIKIRI-GENMAN
 
| YUBIKIRI-GENMAN
Line 68: Line 75:
 
| Japanese
 
| Japanese
 
| j-pop, progressive, female vocal
 
| j-pop, progressive, female vocal
 +
| 59-146 (mostly 146)
 
|-
 
|-
 
| Himitsu feat. KAFU
 
| Himitsu feat. KAFU
Line 74: Line 82:
 
| Japanese
 
| Japanese
 
| j-pop, vocaloid
 
| j-pop, vocaloid
 +
| 168
 +
|}
 +
 +
=== Default Settings ===
 +
 +
{| class="wikitable"
 +
|-
 +
! Parameter
 +
! Kantan
 +
! Futsuu
 +
! Muzukashii
 +
! Oni
 +
! Inner Oni
 +
|-
 +
| Overall Difficulty
 +
| 3
 +
| 4
 +
| 5
 +
| 5
 +
| 6
 +
|-
 +
| HP Drain Rate
 +
| 8
 +
| 7
 +
| 6
 +
| 5
 +
| 5
 
|}
 
|}
  
Line 79: Line 114:
  
 
== Algorithmic Evaluation Results ==
 
== Algorithmic Evaluation Results ==
 +
 +
120 beatmaps were checked using [https://osu.ppy.sh/wiki/en/Client/Beatmap_editor/AiMod AiMod] across 4 models, 6 songs, and 5 difficulties.
 +
 +
'''Difficulty ordering warnings:'''
 +
* DOSU - closer to you - Inner Oni < Oni
 +
* MALer - YUBIKIRI-GENMAN - Inner Oni < Oni
 +
* MALer - closer to you - Kantan < Oni < Futsuu < Inner Oni < Muzukashii
 +
 +
=== DOSU ===
 +
 +
{| class="wikitable"
 +
|-
 +
! Song
 +
! Kantan
 +
! Futsuu
 +
! Muzukashii
 +
! Oni
 +
! Inner Oni
 +
|-
 +
| Silver Bullet
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| A fight in Tunisia
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| Himitsu
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| yubikiri-genman
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| something fun
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| closer to you
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|}
 +
 +
=== FumenLab ===
 +
 +
{| class="wikitable"
 +
|-
 +
! Song
 +
! Kantan
 +
! Futsuu
 +
! Muzukashii
 +
! Oni
 +
! Inner Oni
 +
|-
 +
| Silver Bullet
 +
| Minor snapping issue on drumrolls and swells
 +
| Minor snapping issue on drumrolls and swells
 +
| Minor snapping issue on drumrolls and swells
 +
| Objects <10ms apart and concurrent objects
 +
| Objects <10ms apart and concurrent objects
 +
|-
 +
| A fight in Tunisia
 +
| Minor snapping issue on drumrolls and swells
 +
| Minor snapping issue on drumrolls and swells
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| Himitsu
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| Minor snapping issue on drumrolls and swells
 +
|-
 +
| yubikiri-genman
 +
| OK
 +
| Minor snapping issue on drumrolls and swells
 +
| Minor snapping issue on drumrolls and swells
 +
| OK
 +
| OK
 +
|-
 +
| something fun
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| closer to you
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|}
 +
 +
=== HiTaiko ===
 +
 +
{| class="wikitable"
 +
|-
 +
! Song
 +
! Kantan
 +
! Futsuu
 +
! Muzukashii
 +
! Oni
 +
! Inner Oni
 +
|-
 +
| Silver Bullet
 +
| Minor snapping issue on drumrolls and swells
 +
| Minor snapping issue on drumrolls and swells
 +
| Minor snapping issue on drumrolls and swells
 +
| Minor snapping issue on drumrolls and swells
 +
| OK
 +
|-
 +
| A fight in Tunisia
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| Himitsu
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| yubikiri-genman
 +
| OK
 +
| OK
 +
| OK
 +
| Minor snapping issue on drumrolls and swells
 +
| Minor snapping issue on drumrolls and swells
 +
|-
 +
| something fun
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| closer to you
 +
| Unused audio at the end
 +
| Unused audio at the end
 +
| Unused audio at the end
 +
| Unused audio at the end
 +
| OK
 +
|}
 +
 +
=== MALer ===
 +
 +
{| class="wikitable"
 +
|-
 +
! Song
 +
! Kantan
 +
! Futsuu
 +
! Muzukashii
 +
! Oni
 +
! Inner Oni
 +
|-
 +
| Silver Bullet
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| A fight in Tunisia
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| Himitsu
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| yubikiri-genman
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| something fun
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
| OK
 +
|-
 +
| closer to you
 +
| Unused audio at the end
 +
| Unused audio at the end
 +
| Unused audio at the end
 +
| Unused audio at the end
 +
| Unused audio at the end
 +
|}
  
 
== Expert Evaluation Results ==
 
== Expert Evaluation Results ==
 +
 +
5 osu!taiko mappers evaluated submissions across 5 songs: '''Silver Bullet''', '''A fight in Tunisia''', '''YUBIKIRI-GENMAN''', '''Something Fun''', and '''closer to you'''. Each judge was assigned 3 songs based on their familiarity and interest of the songs, and each song was evaluated by 3 judges. For every assigned song, judges assessed submissions from all 4 models on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10.
 +
 +
We follow [https://osu.ppy.sh/wiki/en/Contests/Monthly_Beatmapping_Contest Monthly Beatmapping Contest] and adopt '''within-song normalization''' for per-song report. For each category, judge X's final score = (judge X's raw score - judge X's average raw score) / judge X's standard deviation within that song, and a submission's score is the sum of each judge's final score.
 +
 +
=== Silver Bullet ===
 +
 +
{| class="wikitable"
 +
! Model
 +
! Musical Representation
 +
! Creativity
 +
! Gameplay
 +
! Human-Likeness
 +
! Qualitative Feedback
 +
|-
 +
| DOSU
 +
| '''1.5872'''
 +
| −2.6304
 +
| '''2.0099'''
 +
| −0.4552
 +
| Musical representation drew mixed assessments. Oni and Muzukashii showed effective sound selection, but Inner Oni was conservative early and uneven near the ending. Inconsistent special-object use and variable-BPM handling weakened coherence across difficulties.
 +
|-
 +
| FumenLab
 +
| 0.3005
 +
| 0.9401
 +
| −3.5863
 +
| −1.9478
 +
| Oni and Inner Oni showed promising sound selection, but awkward bursts and near-overlapping notes compromised gameplay near the ending. Kantan was excessively sparse, while intermediate difficulties were comparatively plain. Repeated musical phrases received inconsistent treatment.
 +
|-
 +
| HiTaiko
 +
| 1.2778
 +
| '''1.8629'''
 +
| '''2.0099'''
 +
| '''4.1634'''
 +
| Coherent musical representation and expressive color patterns were recurring strengths. Inner Oni made quieter passages disproportionately dense, while Oni was comparatively restrained. Futsuu and Muzukashii showed promising structure, although assessments of the lower-tier ending differed. Finisher placement remained inconsistent.
 +
|-
 +
| MALer
 +
| −3.1656
 +
| −0.1726
 +
| −0.4335
 +
| −1.7604
 +
| Imaginative Oni patterns and distinctive sectioning were offset by rhythms that often felt disconnected from the music. Inner Oni contained timing and overlap problems; unusual subdivisions in Muzukashii and questionable sound selection in Futsuu weakened the difficulty spread.
 +
|}
 +
 +
=== A fight in Tunisia ===
 +
 +
{| class="wikitable"
 +
! Model
 +
! Musical Representation
 +
! Creativity
 +
! Gameplay
 +
! Human-Likeness
 +
! Qualitative Feedback
 +
|-
 +
| DOSU
 +
| −4.7037
 +
| −3.6090
 +
| 0.4677
 +
| −1.5881
 +
| The mapping frequently missed prominent musical events across all difficulties. Conservative patterns supported playability, but omitted piano passages and notes in musically empty sections weakened representation. Isolated Inner Oni patterns showed potential without fitting the song.
 +
|-
 +
| FumenLab
 +
| 0.2720
 +
| −0.2284
 +
| −2.4438
 +
| −0.1690
 +
| Awkward bursts, excessive local density and neglected piano passages weakened musical clarity. Kantan–Muzukashii lacked coherent sound selection; Oni was sometimes too demanding for its tier. Inner Oni was relatively stronger but remained rhythmically awkward.
 +
|-
 +
| HiTaiko
 +
| 2.0747
 +
| 0.9776
 +
| '''2.2563'''
 +
| '''2.3546'''
 +
| Oni stood out for coherent sound selection and structure. Inner Oni was more conservative and relied on abrupt doublets, while Kantan–Muzukashii often omitted material or mixed musical layers inconsistently. Piano representation and repeated special-object placement remained weaknesses.
 +
|-
 +
| MALer
 +
| '''2.3570'''
 +
| '''2.8598'''
 +
| −0.2803
 +
| −0.5975
 +
| Expressive color patterns attracted praise, but overall assessments differed. Complex snapping and inconsistent musical-layer selection reduced readability, particularly in Kantan–Muzukashii. Inner Oni offered some musical representation, although its structure and rhythmic emphasis remained uneven.
 +
|}
 +
 +
=== YUBIKIRI-GENMAN ===
 +
 +
{| class="wikitable"
 +
! Model
 +
! Musical Representation
 +
! Creativity
 +
! Gameplay
 +
! Human-Likeness
 +
! Qualitative Feedback
 +
|-
 +
| DOSU
 +
| −1.6143
 +
| '''0.6706'''
 +
| −1.2916
 +
| −1.5202
 +
| Excessive rhythmic improvisation disrupted the song’s gentle character. Muzukashii, Oni and Inner Oni often emphasized verses over choruses, weakening emotional progression. Kantan’s simpler rhythms were more acceptable, but consistency, object placement and musical correspondence remained concerns.
 +
|-
 +
| FumenLab
 +
| 1.3093
 +
| 0.3325
 +
| '''2.5865'''
 +
| 0.6172
 +
| Assessments of musical coherence differed. Muzukashii and Oni sometimes followed the vocals effectively, but repeated phrases received inconsistent treatment. Inner Oni’s complex climax disrupted the gentle atmosphere. Kantan was sparse, and hitsound volume and effect choices required refinement.
 +
|-
 +
| HiTaiko
 +
| '''1.3093'''
 +
| −0.2970
 +
| 1.8815
 +
| '''4.0095'''
 +
| Oni and Inner Oni showed effective vocal following and a convincing build toward the final chorus, although some feedback found them too conservative. Muzukashii’s density allocation was inconsistent; Kantan and Futsuu offered little content. Special-object use and omissions remained concerns.
 +
|-
 +
| MALer
 +
| −1.0043
 +
| −0.7060
 +
| −3.1764
 +
| −3.1065
 +
| Irregular snapping and abrupt density changes were poorly suited to the gentle vocal/piano arrangement. Muzukashii, Oni and Inner Oni contained isolated difficulty spikes, with Oni reportedly harder than Inner Oni. Some color patterns and brief rhythm choices worked but were not sustained.
 +
|}
 +
''Musical Representation scores for FumenLab and HiTaiko round to the same value; HiTaiko has the higher unrounded score.''
 +
 +
=== Something Fun ===
 +
 +
{| class="wikitable"
 +
! Model
 +
! Musical Representation
 +
! Creativity
 +
! Gameplay
 +
! Human-Likeness
 +
! Qualitative Feedback
 +
|-
 +
| DOSU
 +
| −4.1362
 +
| −3.0094
 +
| −2.8967
 +
| −3.9851
 +
| Rhythmic choices and sound selection were poorly aligned with the music. Missing notes, excessive finishers and misplaced spinners weakened consistency. Both Muzukashii and Inner Oni contained passages whose musical intent was unclear.
 +
|-
 +
| FumenLab
 +
| 1.7808
 +
| 2.0497
 +
| '''2.0645'''
 +
| '''2.1521'''
 +
| The set was broadly serviceable, but repetitive color patterns and inconsistent finishers limited expressiveness. Kantan and Futsuu were weak, while some Inner Oni streams added unnecessary complexity. Isolated passages also raised timing concerns.
 +
|-
 +
| HiTaiko
 +
| −1.0707
 +
| −1.3013
 +
| −0.3393
 +
| 0.6620
 +
| Some patterns followed the melody effectively, but prolonged simple rhythms, questionable drumrolls and uneven sectional difficulty made the experience repetitive. Finisher placement lacked consistency; no particular difficulty tier emerged as a clear strength.
 +
|-
 +
| MALer
 +
| '''3.4261'''
 +
| '''2.2610'''
 +
| 1.1716
 +
| 1.1710
 +
| Creative ideas produced sharply divided assessments. Futsuu omitted a conspicuous musical passage, while Inner Oni showed uneven density and timing problems. Kantan and Muzukashii had questionable rhythmic choices; Oni was comparatively more successful but appeared too easy for its label.
 +
|}
 +
 +
=== closer to you ===
 +
 +
{| class="wikitable"
 +
! Model
 +
! Musical Representation
 +
! Creativity
 +
! Gameplay
 +
! Human-Likeness
 +
! Qualitative Feedback
 +
|-
 +
| DOSU
 +
| −0.5579
 +
| 0.7683
 +
| −0.7678
 +
| −0.5907
 +
| Rhythmic improvisation showed promise, and Futsuu conveyed a pleasing feel. However, inconsistent finishers and uneven density weakened progression: Oni omitted the opening, while Inner Oni used fewer streams than Muzukashii.
 +
|-
 +
| FumenLab
 +
| '''3.3983'''
 +
| '''4.2866'''
 +
| '''4.4877'''
 +
| '''3.5659'''
 +
| Strong structure and effective rhythmic improvisation were recurring positives, with Inner Oni receiving particular praise. Color-pattern variety, consistency and timing-point use still needed refinement, but the set conveyed a comparatively convincing mapping approach.
 +
|-
 +
| HiTaiko
 +
| 1.6737
 +
| −2.5275
 +
| −0.1399
 +
| 1.0151
 +
| Patterns were coherent but repetitive, with little distinction between difficulty tiers. Even Inner Oni remained sparse and closer to a much easier difficulty. Drumroll use and the ending finisher were questionable choices for the song’s calm character.
 +
|-
 +
| MALer
 +
| −4.5141
 +
| −2.5275
 +
| −3.5800
 +
| −3.9903
 +
| Missing sections, timing errors and inconsistent musical logic undermined the spread. Muzukashii was reportedly the hardest tier, while easier difficulties contained unexplained breaks. Inner Oni’s awkward fast bursts did not resolve the overall lack of musical coverage.
 +
|}
 +
 +
Qualitative feedback suggested distinct tendencies across the evaluated models. HiTaiko often favored coherent but conservative arrangements, while MALer explored more unusual rhythms with less consistent musical fit. FumenLab demonstrated effective improvisation and higher-difficulty patterning, but struggled with simplification and technical consistency. DOSU showed isolated strengths in sound selection, alongside substantial variation in musical correspondence and structural coherence. Across models, adapting musical ideas to lower difficulties remained a recurring challenge.
  
 
== Community Evaluation Results ==
 
== Community Evaluation Results ==
 +
 +
{| class="wikitable"
 +
|-
 +
! Mapper
 +
! Musical Representation
 +
! Creativity
 +
! Gameplay
 +
! Human-Likeness
 +
! Overall Mean
 +
|-
 +
| DOSU
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
| FumenLab
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
| MALer
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
| HiTaiko
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
| Amasugi (human mapper)
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|}

Latest revision as of 17:15, 12 October 2026

Submissions

Submission Title PDF Authors
DOSU DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION PDF Junyoung Koh, Hao-Wen Dong
FumenLab FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS PDF Ryo Mitoma
HiTaiko HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection PDF Sungjung Park
MALer RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL PDF Sojeong An, Chris Donahue, Dasaem Jeong

Test Set

Song Selection

Title Artist Genre Language Tags BPM
Silver Bullet Ennnn Electronic Instrumental breakcore, neurofunk, tech 89-379 (mostly 175)
A fight in Tunisia Sobrem & NIWASHI Jazz Instrumental piano 148
Something Fun JOYLESS Rock English male vocal 194
closer to you (than ever before) xiiiac13 Electronic Instrumental lo-fi 110
YUBIKIRI-GENMAN Mili Pop Japanese j-pop, progressive, female vocal 59-146 (mostly 146)
Himitsu feat. KAFU MIMI Pop Japanese j-pop, vocaloid 168

Default Settings

Parameter Kantan Futsuu Muzukashii Oni Inner Oni
Overall Difficulty 3 4 5 5 6
HP Drain Rate 8 7 6 5 5

Results

Algorithmic Evaluation Results

120 beatmaps were checked using AiMod across 4 models, 6 songs, and 5 difficulties.

Difficulty ordering warnings:

  • DOSU - closer to you - Inner Oni < Oni
  • MALer - YUBIKIRI-GENMAN - Inner Oni < Oni
  • MALer - closer to you - Kantan < Oni < Futsuu < Inner Oni < Muzukashii

DOSU

Song Kantan Futsuu Muzukashii Oni Inner Oni
Silver Bullet OK OK OK OK OK
A fight in Tunisia OK OK OK OK OK
Himitsu OK OK OK OK OK
yubikiri-genman OK OK OK OK OK
something fun OK OK OK OK OK
closer to you OK OK OK OK OK

FumenLab

Song Kantan Futsuu Muzukashii Oni Inner Oni
Silver Bullet Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells Objects <10ms apart and concurrent objects Objects <10ms apart and concurrent objects
A fight in Tunisia Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells OK OK OK
Himitsu OK OK OK OK Minor snapping issue on drumrolls and swells
yubikiri-genman OK Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells OK OK
something fun OK OK OK OK OK
closer to you OK OK OK OK OK

HiTaiko

Song Kantan Futsuu Muzukashii Oni Inner Oni
Silver Bullet Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells OK
A fight in Tunisia OK OK OK OK OK
Himitsu OK OK OK OK OK
yubikiri-genman OK OK OK Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells
something fun OK OK OK OK OK
closer to you Unused audio at the end Unused audio at the end Unused audio at the end Unused audio at the end OK

MALer

Song Kantan Futsuu Muzukashii Oni Inner Oni
Silver Bullet OK OK OK OK OK
A fight in Tunisia OK OK OK OK OK
Himitsu OK OK OK OK OK
yubikiri-genman OK OK OK OK OK
something fun OK OK OK OK OK
closer to you Unused audio at the end Unused audio at the end Unused audio at the end Unused audio at the end Unused audio at the end

Expert Evaluation Results

5 osu!taiko mappers evaluated submissions across 5 songs: Silver Bullet, A fight in Tunisia, YUBIKIRI-GENMAN, Something Fun, and closer to you. Each judge was assigned 3 songs based on their familiarity and interest of the songs, and each song was evaluated by 3 judges. For every assigned song, judges assessed submissions from all 4 models on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10.

We follow Monthly Beatmapping Contest and adopt within-song normalization for per-song report. For each category, judge X's final score = (judge X's raw score - judge X's average raw score) / judge X's standard deviation within that song, and a submission's score is the sum of each judge's final score.

Silver Bullet

Model Musical Representation Creativity Gameplay Human-Likeness Qualitative Feedback
DOSU 1.5872 −2.6304 2.0099 −0.4552 Musical representation drew mixed assessments. Oni and Muzukashii showed effective sound selection, but Inner Oni was conservative early and uneven near the ending. Inconsistent special-object use and variable-BPM handling weakened coherence across difficulties.
FumenLab 0.3005 0.9401 −3.5863 −1.9478 Oni and Inner Oni showed promising sound selection, but awkward bursts and near-overlapping notes compromised gameplay near the ending. Kantan was excessively sparse, while intermediate difficulties were comparatively plain. Repeated musical phrases received inconsistent treatment.
HiTaiko 1.2778 1.8629 2.0099 4.1634 Coherent musical representation and expressive color patterns were recurring strengths. Inner Oni made quieter passages disproportionately dense, while Oni was comparatively restrained. Futsuu and Muzukashii showed promising structure, although assessments of the lower-tier ending differed. Finisher placement remained inconsistent.
MALer −3.1656 −0.1726 −0.4335 −1.7604 Imaginative Oni patterns and distinctive sectioning were offset by rhythms that often felt disconnected from the music. Inner Oni contained timing and overlap problems; unusual subdivisions in Muzukashii and questionable sound selection in Futsuu weakened the difficulty spread.

A fight in Tunisia

Model Musical Representation Creativity Gameplay Human-Likeness Qualitative Feedback
DOSU −4.7037 −3.6090 0.4677 −1.5881 The mapping frequently missed prominent musical events across all difficulties. Conservative patterns supported playability, but omitted piano passages and notes in musically empty sections weakened representation. Isolated Inner Oni patterns showed potential without fitting the song.
FumenLab 0.2720 −0.2284 −2.4438 −0.1690 Awkward bursts, excessive local density and neglected piano passages weakened musical clarity. Kantan–Muzukashii lacked coherent sound selection; Oni was sometimes too demanding for its tier. Inner Oni was relatively stronger but remained rhythmically awkward.
HiTaiko 2.0747 0.9776 2.2563 2.3546 Oni stood out for coherent sound selection and structure. Inner Oni was more conservative and relied on abrupt doublets, while Kantan–Muzukashii often omitted material or mixed musical layers inconsistently. Piano representation and repeated special-object placement remained weaknesses.
MALer 2.3570 2.8598 −0.2803 −0.5975 Expressive color patterns attracted praise, but overall assessments differed. Complex snapping and inconsistent musical-layer selection reduced readability, particularly in Kantan–Muzukashii. Inner Oni offered some musical representation, although its structure and rhythmic emphasis remained uneven.

YUBIKIRI-GENMAN

Model Musical Representation Creativity Gameplay Human-Likeness Qualitative Feedback
DOSU −1.6143 0.6706 −1.2916 −1.5202 Excessive rhythmic improvisation disrupted the song’s gentle character. Muzukashii, Oni and Inner Oni often emphasized verses over choruses, weakening emotional progression. Kantan’s simpler rhythms were more acceptable, but consistency, object placement and musical correspondence remained concerns.
FumenLab 1.3093 0.3325 2.5865 0.6172 Assessments of musical coherence differed. Muzukashii and Oni sometimes followed the vocals effectively, but repeated phrases received inconsistent treatment. Inner Oni’s complex climax disrupted the gentle atmosphere. Kantan was sparse, and hitsound volume and effect choices required refinement.
HiTaiko 1.3093 −0.2970 1.8815 4.0095 Oni and Inner Oni showed effective vocal following and a convincing build toward the final chorus, although some feedback found them too conservative. Muzukashii’s density allocation was inconsistent; Kantan and Futsuu offered little content. Special-object use and omissions remained concerns.
MALer −1.0043 −0.7060 −3.1764 −3.1065 Irregular snapping and abrupt density changes were poorly suited to the gentle vocal/piano arrangement. Muzukashii, Oni and Inner Oni contained isolated difficulty spikes, with Oni reportedly harder than Inner Oni. Some color patterns and brief rhythm choices worked but were not sustained.

Musical Representation scores for FumenLab and HiTaiko round to the same value; HiTaiko has the higher unrounded score.

Something Fun

Model Musical Representation Creativity Gameplay Human-Likeness Qualitative Feedback
DOSU −4.1362 −3.0094 −2.8967 −3.9851 Rhythmic choices and sound selection were poorly aligned with the music. Missing notes, excessive finishers and misplaced spinners weakened consistency. Both Muzukashii and Inner Oni contained passages whose musical intent was unclear.
FumenLab 1.7808 2.0497 2.0645 2.1521 The set was broadly serviceable, but repetitive color patterns and inconsistent finishers limited expressiveness. Kantan and Futsuu were weak, while some Inner Oni streams added unnecessary complexity. Isolated passages also raised timing concerns.
HiTaiko −1.0707 −1.3013 −0.3393 0.6620 Some patterns followed the melody effectively, but prolonged simple rhythms, questionable drumrolls and uneven sectional difficulty made the experience repetitive. Finisher placement lacked consistency; no particular difficulty tier emerged as a clear strength.
MALer 3.4261 2.2610 1.1716 1.1710 Creative ideas produced sharply divided assessments. Futsuu omitted a conspicuous musical passage, while Inner Oni showed uneven density and timing problems. Kantan and Muzukashii had questionable rhythmic choices; Oni was comparatively more successful but appeared too easy for its label.

closer to you

Model Musical Representation Creativity Gameplay Human-Likeness Qualitative Feedback
DOSU −0.5579 0.7683 −0.7678 −0.5907 Rhythmic improvisation showed promise, and Futsuu conveyed a pleasing feel. However, inconsistent finishers and uneven density weakened progression: Oni omitted the opening, while Inner Oni used fewer streams than Muzukashii.
FumenLab 3.3983 4.2866 4.4877 3.5659 Strong structure and effective rhythmic improvisation were recurring positives, with Inner Oni receiving particular praise. Color-pattern variety, consistency and timing-point use still needed refinement, but the set conveyed a comparatively convincing mapping approach.
HiTaiko 1.6737 −2.5275 −0.1399 1.0151 Patterns were coherent but repetitive, with little distinction between difficulty tiers. Even Inner Oni remained sparse and closer to a much easier difficulty. Drumroll use and the ending finisher were questionable choices for the song’s calm character.
MALer −4.5141 −2.5275 −3.5800 −3.9903 Missing sections, timing errors and inconsistent musical logic undermined the spread. Muzukashii was reportedly the hardest tier, while easier difficulties contained unexplained breaks. Inner Oni’s awkward fast bursts did not resolve the overall lack of musical coverage.

Qualitative feedback suggested distinct tendencies across the evaluated models. HiTaiko often favored coherent but conservative arrangements, while MALer explored more unusual rhythms with less consistent musical fit. FumenLab demonstrated effective improvisation and higher-difficulty patterning, but struggled with simplification and technical consistency. DOSU showed isolated strengths in sound selection, alongside substantial variation in musical correspondence and structural coherence. Across models, adapting musical ideas to lower difficulties remained a recurring challenge.

Community Evaluation Results

Mapper Musical Representation Creativity Gameplay Human-Likeness Overall Mean
DOSU
FumenLab
MALer
HiTaiko
Amasugi (human mapper)
Retrieved from "https://music-ir.org/mirex/w/index.php?title=2026:Rhythm_Game_Chart_Generation_Results&oldid=15144"