Difference between revisions of "2026:Rhythm Game Chart Generation Results"

From MIREX Wiki
(→‎Expert Evaluation Results)
(→‎Silver Bullet)
Line 350: Line 350:
 
! Gameplay
 
! Gameplay
 
! Human-Likeness
 
! Human-Likeness
 +
! Qualitative Feedback
 
|-
 
|-
 
| DOSU
 
| DOSU
Line 356: Line 357:
 
| '''2.0099'''
 
| '''2.0099'''
 
| −0.4552
 
| −0.4552
 +
| Musical representation drew mixed assessments. Oni and Muzukashii showed effective sound selection, but Inner Oni was conservative early and uneven near the ending. Inconsistent special-object use and variable-BPM handling weakened coherence across difficulties.
 
|-
 
|-
 
| FumenLab
 
| FumenLab
Line 362: Line 364:
 
| −3.5863
 
| −3.5863
 
| −1.9478
 
| −1.9478
 +
| Oni and Inner Oni showed promising sound selection, but awkward bursts and near-overlapping notes compromised gameplay near the ending. Kantan was excessively sparse, while intermediate difficulties were comparatively plain. Repeated musical phrases received inconsistent treatment.
 
|-
 
|-
 
| HiTaiko
 
| HiTaiko
Line 368: Line 371:
 
| '''2.0099'''
 
| '''2.0099'''
 
| '''4.1634'''
 
| '''4.1634'''
 +
| Coherent musical representation and expressive color patterns were recurring strengths. Inner Oni made quieter passages disproportionately dense, while Oni was comparatively restrained. Futsuu and Muzukashii showed promising structure, although assessments of the lower-tier ending differed. Finisher placement remained inconsistent.
 
|-
 
|-
 
| MALer
 
| MALer
Line 374: Line 378:
 
| −0.4335
 
| −0.4335
 
| −1.7604
 
| −1.7604
 +
| Imaginative Oni patterns and distinctive sectioning were offset by rhythms that often felt disconnected from the music. Inner Oni contained timing and overlap problems; unusual subdivisions in Muzukashii and questionable sound selection in Futsuu weakened the difficulty spread.
 
|}
 
|}
  

Revision as of 09:28, 11 October 2026

Submissions

Submission Title PDF Authors
DOSU DOSU: DIFFICULTY-CONDITIONED ON-GRID SEQUENCE UNMASKING FOR OSU!TAIKO CHART GENERATION Junyoung Koh, Hao-Wen Dong
FumenLab FUMENLAB: PLAN-CONDITIONED JOINT GENERATION AND REFINEMENT OF OSU!TAIKO CHARTS Ryo Mitoma
HiTaiko HiTaiko: Hierarchical osu!Taiko Chart Generation via Metrical Candidate Selection Sungjung Park
MALer RHYTHM GAME CHART GENERATION WITH PRE-TRAINED MUSIC TRANSCRIPTION MODEL Sojeong An, Chris Donahue, Dasaem Jeong

Test Set

Song Selection

Title Artist Genre Language Tags BPM
Silver Bullet Ennnn Electronic Instrumental breakcore, neurofunk, tech 89-379 (mostly 175)
A fight in Tunisia Sobrem & NIWASHI Jazz Instrumental piano 148
Something Fun JOYLESS Rock English male vocal 194
closer to you (than ever before) xiiiac13 Electronic Instrumental lo-fi 110
YUBIKIRI-GENMAN Mili Pop Japanese j-pop, progressive, female vocal 59-146 (mostly 146)
Himitsu feat. KAFU MIMI Pop Japanese j-pop, vocaloid 168

Default Settings

Parameter Kantan Futsuu Muzukashii Oni Inner Oni
Overall Difficulty 3 4 5 5 6
HP Drain Rate 8 7 6 5 5

Results

Algorithmic Evaluation Results

120 beatmaps were checked using AiMod across 4 models, 6 songs, and 5 difficulties.

Difficulty ordering warnings:

  • DOSU - closer to you - Inner Oni < Oni
  • MALer - YUBIKIRI-GENMAN - Inner Oni < Oni
  • MALer - closer to you - Kantan < Oni < Futsuu < Inner Oni < Muzukashii

DOSU

Song Kantan Futsuu Muzukashii Oni Inner Oni
Silver Bullet OK OK OK OK OK
A fight in Tunisia OK OK OK OK OK
Himitsu OK OK OK OK OK
yubikiri-genman OK OK OK OK OK
something fun OK OK OK OK OK
closer to you OK OK OK OK OK

FumenLab

Song Kantan Futsuu Muzukashii Oni Inner Oni
Silver Bullet Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells Objects <10ms apart and concurrent objects Objects <10ms apart and concurrent objects
A fight in Tunisia Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells OK OK OK
Himitsu OK OK OK OK Minor snapping issue on drumrolls and swells
yubikiri-genman OK Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells OK OK
something fun OK OK OK OK OK
closer to you OK OK OK OK OK

HiTaiko

Song Kantan Futsuu Muzukashii Oni Inner Oni
Silver Bullet Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells OK
A fight in Tunisia OK OK OK OK OK
Himitsu OK OK OK OK OK
yubikiri-genman OK OK OK Minor snapping issue on drumrolls and swells Minor snapping issue on drumrolls and swells
something fun OK OK OK OK OK
closer to you Unused audio at the end Unused audio at the end Unused audio at the end Unused audio at the end OK

MALer

Song Kantan Futsuu Muzukashii Oni Inner Oni
Silver Bullet OK OK OK OK OK
A fight in Tunisia OK OK OK OK OK
Himitsu OK OK OK OK OK
yubikiri-genman OK OK OK OK OK
something fun OK OK OK OK OK
closer to you Unused audio at the end Unused audio at the end Unused audio at the end Unused audio at the end Unused audio at the end

Expert Evaluation Results

5 osu!taiko mappers evaluated submissions across 5 songs: Silver Bullet, A fight in Tunisia, YUBIKIRI-GENMAN, Something Fun, and closer to you. Each judge was assigned 3 songs based on their familiarity and interest of the songs, and each song was evaluated by 3 judges. For every assigned song, judges assessed submissions from all 4 models on 4 categories (Musical Representation, Creativity, Gameplay, and Human-Likeness) using an integer scale from 1 to 10. We follow Monthly Beatmapping Contest and adopt within-song normalization. For each category, judge X's final score = (judge X's raw score - judge X's average raw score) / judge X's standard deviation within that song, and a submission's score is the sum of each judge's final score.

Silver Bullet

Model Musical Representation Creativity Gameplay Human-Likeness Qualitative Feedback
DOSU 1.5872 −2.6304 2.0099 −0.4552 Musical representation drew mixed assessments. Oni and Muzukashii showed effective sound selection, but Inner Oni was conservative early and uneven near the ending. Inconsistent special-object use and variable-BPM handling weakened coherence across difficulties.
FumenLab 0.3005 0.9401 −3.5863 −1.9478 Oni and Inner Oni showed promising sound selection, but awkward bursts and near-overlapping notes compromised gameplay near the ending. Kantan was excessively sparse, while intermediate difficulties were comparatively plain. Repeated musical phrases received inconsistent treatment.
HiTaiko 1.2778 1.8629 2.0099 4.1634 Coherent musical representation and expressive color patterns were recurring strengths. Inner Oni made quieter passages disproportionately dense, while Oni was comparatively restrained. Futsuu and Muzukashii showed promising structure, although assessments of the lower-tier ending differed. Finisher placement remained inconsistent.
MALer −3.1656 −0.1726 −0.4335 −1.7604 Imaginative Oni patterns and distinctive sectioning were offset by rhythms that often felt disconnected from the music. Inner Oni contained timing and overlap problems; unusual subdivisions in Muzukashii and questionable sound selection in Futsuu weakened the difficulty spread.

Community Evaluation Results

Mapper Musical Representation Creativity Gameplay Human-Likeness Overall Mean
DOSU
FumenLab
MALer
HiTaiko
Amasugi (human mapper)
Retrieved from "https://music-ir.org/mirex/w/index.php?title=2026:Rhythm_Game_Chart_Generation_Results&oldid=15129"