Difference between revisions of "2026:Rhythm Game Chart Generation"

From MIREX Wiki
(→‎Resources)
Line 98: Line 98:
  
  
−
= Long-term plan =
+
= Long-term Plan =
  
 
We are committed to maintaining and expanding this task in the coming years. Our long-term roadmap includes the introduction of additional osu! modes and other open-access rhythm games, as well as other research objectives such as cross-mode/cross-game transfer learning ability and fine-grained generation controllability.
 
We are committed to maintaining and expanding this task in the coming years. Our long-term roadmap includes the introduction of additional osu! modes and other open-access rhythm games, as well as other research objectives such as cross-mode/cross-game transfer learning ability and fine-grained generation controllability.

Revision as of 15:17, 6 June 2026

Task Description

Evaluation Criteria

While existing research relies on researcher-defined quantitative evaluation methods, we adopt a more user-centered 3-stage approach. The following evaluation will be performed on songs that are not included in the dataset provided to the participants. We will recruit about 5 osu!taiko mappers as the jury, and the jury will also be involved in the test song selection process.

Algorithmic Evaluation

In this stage, we will manually pick 3 songs for evaluation. First, we will use osu! star rating calculator to calculate the star ratings of the generated beatmaps. This allows us to assess whether the model's difficulty conditioning is functioning correctly. Then, we will follow AiMod and MapsetVerifier to detect unplayable placements of objects algorithmically. Models passing this evaluation phase will move on to human evaluation.

Expert Evaluation

In this stage, the jury will rate the generated charts based on musical representation, creativity, and gameplay similar to the Monthly Beatmapping Contest, and we will also introduce a humanity score evaluating how human-like the generated charts are. We will then use the jury's ratings to determine which finalists move on to player evaluation.

Community Evaluation

In this stage, a single showcase song will be selected for community evaluation. Participants will choose a difficulty level aligned with their skill level and playtest charts generated by each model alongside a human-authored chart. Players will vote on musical representation, creativity, gameplay, and they will be asked to identify which chart is authored by human.

Resources

We recommend participants to read this document to get an overall idea of how osu!taiko mapping works from a mapper's perspective. This is especially helpful if you are not familiar with osu!taiko gaming and/or mapping.

Participants are also encouraged to read osu! wiki to gain a clearer understanding of the gameplay, data format, timing, and mapping goals. We hereby list some very useful pages:

List of popular GitHub repositories for osu! beatmap generation models without papers:

List of papers and extended abstracts on osu! beatmap generation:

  • Hanzen, Jonah, Emily Halina, and Matthew Guzdial. "Time-based Chart Partitioning: Improving Local Coherency in Rhythm Game Chart Generation." Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment. Vol. 21. No. 1. 2025.
  • Yi, Jayeon, Sungho Lee, and Kyogu Lee. "Beat-Aligned Spectrogram-to-Sequence Generation of Rhythm-Game Charts." ISMIR LBD. 2023. GitHub repo: https://github.com/stet-stet/goct_ismir2023
  • Lee, Sihun, and Dasaem Jeong. "AutoOsu: Audio-Aware Action Generation for Rhythm Games." ISMIR LBD. 2023. GitHub repo: https://github.com/issyun/AutoOsu
  • Halina, Emily, and Matthew Guzdial. "TaikoNation: Patterning-focused chart generation for rhythm action games." Proceedings of the 16th International Conference on the Foundations of Digital Games. 2021. GitHub repo: https://github.com/emily-halina/TaikoNationV1

Training Dataset

To ensure data quality and fairness, we require all participants to use beatmaps ranked from May 5, 2019 to December 31, 2025 for training and validation. We recommend using this website and filter for "ranked>=2019-05-05 and ranked<2026-01-01 and mode=t", which yields 727 beatmapsets. To minimize legal risks and save resources, we also encourage participants to alternatively filter for "ranked>=2019-05-05 and ranked<2026-01-01 and mode=t and featured artist", which yields 174 beatmapsets. For more information on featured artists, see this page.

In osu!taiko, there are usually Kantan, Futsuu, Muzukashii, Oni, and Inner Oni difficulty levels. However, sometimes people may use different names for them, and there also exist levels that are higher than Inner Oni. To ensure consistency, we recommend using the following tier classification logic: 

   raw_name = osu_file.version_name.lower()
   
   # Keep only a-z and '
   clean_name = RegexReplace(raw_name, "[^a-z']", "") 
   
   if "kantan" in clean_name:
       return 1
       
   elif "futsuu" in clean_name:
       return 2
       
   elif "muzukashii" in clean_name:
       return 3
       
   elif "oni" in clean_name:
       if "inner" in clean_name or "ura" in clean_name:
           return 5
           
       elif "hell" in clean_name:
           return 6
           
       elif clean_name == "oni" or clean_name matches "^.+'soni$":
           return 4
           
   return None  # Discard non-standard variants

Submission Requirements

Timeline

  • July 1, 2026: Submission system opens
  • Sep 11, 2026: Submission deadline for the generation system
  • Oct 15, 2026: Results published

Contact


Long-term Plan

We are committed to maintaining and expanding this task in the coming years. Our long-term roadmap includes the introduction of additional osu! modes and other open-access rhythm games, as well as other research objectives such as cross-mode/cross-game transfer learning ability and fine-grained generation controllability.

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2026:Rhythm_Game_Chart_Generation&oldid=14917"