Difference between revisions of "2025:RenCon Results"

From MIREX Wiki
 
(2 intermediate revisions by 2 users not shown)
Line 33: Line 33:
  
 
The following table shows the final rankings based on weighted average scores from the preliminary round evaluation:
 
The following table shows the final rankings based on weighted average scores from the preliminary round evaluation:
−
 
 
{| class="wikitable sortable"
 
{| class="wikitable sortable"
 
! Rank
 
! Rank
Line 40: Line 39:
 
! Authors/Institution
 
! Authors/Institution
 
! Weighted Score
 
! Weighted Score
−
! Simple Average
+
! PDF
−
! Responses
 
 
|-
 
|-
 
| 1
 
| 1
−
| EmberSky
+
| MidnightOpal
−
| [System Name]
+
| DirectorMusices
−
| [Author Names]
+
| Anders Friberg, Gabriel Jones
−
| [X.XXX]/5.0
+
| 4.33/5.0
−
| [X.XX]/5.0
+
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/DirectorMusices.pdf PDF]
−
| 24
 
 
|-
 
|-
 
| 2
 
| 2
−
| AzureThunder
+
| CrystalEcho
−
| [System Name]
+
| VirtuosoNet
−
| [Author Names]
+
| Dasaem Jeong, Taegyun Kwon, Yoojin Kim, Kyogu Lee, Juhan Nam
−
| [X.XXX]/5.0
+
| 3.54/5.0
−
| [X.XX]/5.0
+
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/VirtuosoNet.pdf PDF]
−
| 24
 
 
|-
 
|-
 
| 3
 
| 3
−
| CrimsonDawn
+
| FrozenRiver
−
| [System Name]
+
| Midihum
−
| [Author Names]
+
| Erich Grunewald
−
| [X.XXX]/5.0
+
| 3.32/5.0
−
| [X.XX]/5.0
+
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/Midihum.pdf PDF]
−
| 24
 
 
|-
 
|-
 
| 4
 
| 4
−
| SilverWave
+
| VelvetStorm
−
| [System Name]
+
| ElegantAIPianist
−
| [Author Names]
+
| Leduo Chen, Xinrui Su, Yuqiang Li, Honyu Andy Shing, Junchuan Zhao, Zihan Chai, Kunyang Zhang, Shengchen Li
−
| [X.XXX]/5.0
+
| 3.19/5.0
−
| [X.XX]/5.0
+
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/ElegantAIPianist.pdf PDF]
−
| 24
 
 
|-
 
|-
 
| 5
 
| 5
−
| VelvetStorm
+
| SilverWave
−
| [System Name]
+
| Contin-U
−
| [Author Names]
+
| Jongmin Jung, Dongmin Kim, Sihun Lee, Seola Cho, Hyungjoon Soh, Irmak Bukey, Chris Donahue, Dasaem Jeong
−
| [X.XXX]/5.0
+
| 3.00/5.0
−
| [X.XX]/5.0
+
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/Contin-U.pdf PDF]
−
| 24
 
 
|-
 
|-
 
| 6
 
| 6
 +
| EmberSky
 +
| YQX+
 +
| Jinwen Zhou, Yuncong Xie, Haochen Wang, Huan Zhang, Aidan Hogg, Simon Dixon
 +
| 2.83/5.0
 +
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/YQX+.pdf PDF]
 +
|-
 +
|
 +
| CrimsonDawn
 +
| ScorePerLockNAR
 +
| Weixi Zhai
 +
| 2.53/5.0
 +
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/ScorePerLockNAR.pdf PDF]
 +
|-
 +
|
 +
| AzureThunder
 +
| RenConnoisseur
 +
| Silvan Peter
 +
| 2.53/5.0
 +
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/RenConnoisseur.pdf PDF]
 +
|-
 +
|
 
| GoldenMist
 
| GoldenMist
−
| [System Name]
+
| CueFreeExpressPedal
−
| [Author Names]
+
| Kyle Worrall, Tom Collins
−
| [X.XXX]/5.0
+
| 2.31/5.0
−
| [X.XX]/5.0
+
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/CueFreeExpressPedal.pdf PDF]
−
| 24
 
 
|}
 
|}
  
Line 108: Line 121:
 
* Handling of complex rhythmic patterns
 
* Handling of complex rhythmic patterns
 
* Balance between technical accuracy and musical expression
 
* Balance between technical accuracy and musical expression
 +
  
 
== Live Contest Results ==
 
== Live Contest Results ==
  
−
''[To be updated following the live contest on September 25, 2025]''
+
The live contest was held on September 25, 2025, at the Daejeon Convention Center during ISMIR 2025. All systems rendered a surprise piece in real-time, with audience voting determining the final rankings. A total of '''50 evaluators''' participated in the live evaluation.
  
 
=== Surprise Piece ===
 
=== Surprise Piece ===
−
* '''Title:''' [To be announced]
+
* '''Title:''' Variation on "Mother and Sister"
−
* '''Composer:''' [To be announced]
+
* '''Composer:''' Hayeon Bang
−
* '''Duration:''' [X minutes]
+
* '''Duration:''' ~3 minutes
−
* '''Style:''' [Musical characteristics]
+
 
 +
 
 +
=== Live Contest Rankings ===
 +
 
 +
{| class="wikitable sortable"
 +
! Rank
 +
! System Index
 +
! Model Name
 +
! Authors
 +
! Live Score
 +
! Preliminary Rank
 +
! Score Change
 +
|-
 +
| 1st
 +
| #8
 +
| VirtuosoNet
 +
| Dasaem Jeong et al.
 +
| 3.62/5.0
 +
| 2nd
 +
| ↑1
 +
|-
 +
| 2nd
 +
| #1
 +
| DirectorMusices
 +
| Anders Friberg et al.
 +
| 3.06/5.0
 +
| 1st
 +
| ↓1
 +
|-
 +
| 3rd
 +
| #2
 +
| Midihum
 +
| Erich Grunewald
 +
| 2.90/5.0
 +
| 3rd
 +
| —
 +
|-
 +
| 4th
 +
| #4
 +
| Contin-U
 +
| Jongmin Jung et al.
 +
| 2.90/5.0
 +
| 5th
 +
| ↑1
 +
|-
 +
| 5th
 +
| #6
 +
| ScorePerLockNAR
 +
| Weixi Zhai
 +
| 2.52/5.0
 +
| 7th
 +
| ↑2
 +
|-
 +
| 6th
 +
| #3
 +
| RenConnoisseur
 +
| Silvan Peter
 +
| 2.40/5.0
 +
| 8th
 +
| ↑2
 +
|-
 +
| 7th
 +
| #9
 +
| ElegantAIPianist
 +
| Leduo Chen et al.
 +
| 2.08/5.0
 +
| 4th
 +
| ↓3
 +
|-
 +
| 8th
 +
| #5
 +
| YQX+
 +
| Jinwen Zhou et al.
 +
| 1.79/5.0
 +
| 6th
 +
| ↓2
 +
|}
 +
 
 +
=== Human Baseline Performance ===
 +
A human performance of the same surprise piece was also evaluated by the audience as a reference point:
 +
 
 +
{| class="wikitable"
 +
! Performance Type
 +
! Score
 +
! Rank Among All Performances
 +
|-
 +
| Human Performance
 +
| 4.40/5.0
 +
| 1st (Overall)
 +
|}
 +
 
 +
=== Performance Analysis ===
 +
 
 +
'''Top Performers:'''
 +
* '''VirtuosoNet''' achieved the highest AI system score (3.62/5.0), improving from 2nd place in the preliminary round
 +
* '''DirectorMusices''' maintained strong performance (3.06/5.0) but dropped from 1st to 2nd place
 +
* '''Midihum''' showed consistency, maintaining 3rd place across both rounds
 +
 
 +
'''Notable Changes:'''
 +
* '''ScorePerLockNAR''' and '''RenConnoisseur''' both improved by 2 positions in the live contest
 +
* '''ElegantAIPianist''' experienced the largest drop, falling from 4th to 7th place
 +
* '''Contin-U''' improved from 5th to 4th place in live evaluation
 +
 
 +
'''Human vs. AI Performance:'''
 +
* The human performance scored 4.40/5.0, significantly higher than the top AI system
 +
* This represents a performance gap of 0.78 points between human and best AI performance
 +
* All AI systems scored below the human baseline, indicating continued room for improvement
 +
 
 +
=== Final Winner ===
 +
 
 +
'''🏆 RenCon 2025 Winner: VirtuosoNet'''
 +
* '''Team:''' Dasaem Jeong, Taegyun Kwon, Yoojin Kim, Kyogu Lee, Juhan Nam
 +
* '''Final Score:''' 3.62/5.0
 +
* '''Achievement:''' Highest-scoring AI system in live contest evaluation
  
−
=== Live Performance Rankings ===
+
The winner was announced at the closing ceremony of ISMIR 2025 on September 25, 2025.
−
''[Results pending live audience voting]''
 
  
−
=== Winner Announcement ===
 
−
''[To be announced at the conclusion of ISMIR 2025]''
 
  
  

Latest revision as of 10:22, 27 September 2025

2025:RenCon Results

Preliminary (Audition) Round Results

Evaluation Methodology

The preliminary round was evaluated through an online listening test with 25 expert evaluators. The evaluation used a weighted voting system where participants self-rated their expertise level from 1-5 stars, with responses weighted accordingly.

Participant Demographics

Our evaluation panel consisted of highly qualified judges:

Expertise Distribution:

  • Expert evaluators (5 stars): 7 participants (29.2%)
  • High confidence (4 stars): 5 participants (20.8%)
  • Moderate confidence (3 stars): 10 participants (41.7%)
  • Lower confidence (1-2 stars): 2 participants (8.4%)
  • Average expertise weight: 3.67/5.0

Professional Background:

  • Music researchers: 12 (54.5%)
  • Music technologists: 10 (45.5%)
  • Active performers: 8 (36.4%)
  • Conservatory students: 6 (27.3%)
  • Music lovers: 15 (68.2%)
  • Concert-goers: 8 (36.4%)

Musical Experience:

  • Strong representation of classical music expertise
  • Diverse musical preferences spanning classical, jazz, pop, and rock
  • Substantial piano experience among evaluators
  • Mix of academic researchers and practicing musicians

System Rankings

The following table shows the final rankings based on weighted average scores from the preliminary round evaluation:

Rank Anonymous Name Real System Name Authors/Institution Weighted Score PDF
1 MidnightOpal DirectorMusices Anders Friberg, Gabriel Jones 4.33/5.0 PDF
2 CrystalEcho VirtuosoNet Dasaem Jeong, Taegyun Kwon, Yoojin Kim, Kyogu Lee, Juhan Nam 3.54/5.0 PDF
3 FrozenRiver Midihum Erich Grunewald 3.32/5.0 PDF
4 VelvetStorm ElegantAIPianist Leduo Chen, Xinrui Su, Yuqiang Li, Honyu Andy Shing, Junchuan Zhao, Zihan Chai, Kunyang Zhang, Shengchen Li 3.19/5.0 PDF
5 SilverWave Contin-U Jongmin Jung, Dongmin Kim, Sihun Lee, Seola Cho, Hyungjoon Soh, Irmak Bukey, Chris Donahue, Dasaem Jeong 3.00/5.0 PDF
6 EmberSky YQX+ Jinwen Zhou, Yuncong Xie, Haochen Wang, Huan Zhang, Aidan Hogg, Simon Dixon 2.83/5.0 PDF
CrimsonDawn ScorePerLockNAR Weixi Zhai 2.53/5.0 PDF
AzureThunder RenConnoisseur Silvan Peter 2.53/5.0 PDF
GoldenMist CueFreeExpressPedal Kyle Worrall, Tom Collins 2.31/5.0 PDF

Note: Complete rankings and system details will be updated following the live contest and final results announcement.

Qualitative Feedback

Evaluators provided extensive qualitative feedback on the systems' performances:

Common Positive Attributes:

  • Natural expressiveness and human-like phrasing
  • Appropriate tempo variations and rubato
  • Musical sensitivity to harmonic structure
  • Dynamic expression and articulation

Areas for Improvement:

  • Consistency across different musical styles
  • Handling of complex rhythmic patterns
  • Balance between technical accuracy and musical expression


Live Contest Results

The live contest was held on September 25, 2025, at the Daejeon Convention Center during ISMIR 2025. All systems rendered a surprise piece in real-time, with audience voting determining the final rankings. A total of 50 evaluators participated in the live evaluation.

Surprise Piece

  • Title: Variation on "Mother and Sister"
  • Composer: Hayeon Bang
  • Duration: ~3 minutes


Live Contest Rankings

Rank System Index Model Name Authors Live Score Preliminary Rank Score Change
1st #8 VirtuosoNet Dasaem Jeong et al. 3.62/5.0 2nd ↑1
2nd #1 DirectorMusices Anders Friberg et al. 3.06/5.0 1st ↓1
3rd #2 Midihum Erich Grunewald 2.90/5.0 3rd —
4th #4 Contin-U Jongmin Jung et al. 2.90/5.0 5th ↑1
5th #6 ScorePerLockNAR Weixi Zhai 2.52/5.0 7th ↑2
6th #3 RenConnoisseur Silvan Peter 2.40/5.0 8th ↑2
7th #9 ElegantAIPianist Leduo Chen et al. 2.08/5.0 4th ↓3
8th #5 YQX+ Jinwen Zhou et al. 1.79/5.0 6th ↓2

Human Baseline Performance

A human performance of the same surprise piece was also evaluated by the audience as a reference point:

Performance Type Score Rank Among All Performances
Human Performance 4.40/5.0 1st (Overall)

Performance Analysis

Top Performers:

  • VirtuosoNet achieved the highest AI system score (3.62/5.0), improving from 2nd place in the preliminary round
  • DirectorMusices maintained strong performance (3.06/5.0) but dropped from 1st to 2nd place
  • Midihum showed consistency, maintaining 3rd place across both rounds

Notable Changes:

  • ScorePerLockNAR and RenConnoisseur both improved by 2 positions in the live contest
  • ElegantAIPianist experienced the largest drop, falling from 4th to 7th place
  • Contin-U improved from 5th to 4th place in live evaluation

Human vs. AI Performance:

  • The human performance scored 4.40/5.0, significantly higher than the top AI system
  • This represents a performance gap of 0.78 points between human and best AI performance
  • All AI systems scored below the human baseline, indicating continued room for improvement

Final Winner

🏆 RenCon 2025 Winner: VirtuosoNet

  • Team: Dasaem Jeong, Taegyun Kwon, Yoojin Kim, Kyogu Lee, Juhan Nam
  • Final Score: 3.62/5.0
  • Achievement: Highest-scoring AI system in live contest evaluation

The winner was announced at the closing ceremony of ISMIR 2025 on September 25, 2025.



External Links

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2025:RenCon_Results&oldid=14841"