Difference between revisions of "2025:RenCon Results"

From MIREX Wiki
(Blanked the page)
(Tag: Blanking)
 
(3 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
= 2025:RenCon Results =
  
 +
== Preliminary (Audition) Round Results ==
 +
 +
=== Evaluation Methodology ===
 +
The preliminary round was evaluated through an online listening test with '''25 expert evaluators'''. The evaluation used a weighted voting system where participants self-rated their expertise level from 1-5 stars, with responses weighted accordingly.
 +
 +
=== Participant Demographics ===
 +
Our evaluation panel consisted of highly qualified judges:
 +
 +
'''Expertise Distribution:'''
 +
* Expert evaluators (5 stars): 7 participants (29.2%)
 +
* High confidence (4 stars): 5 participants (20.8%)
 +
* Moderate confidence (3 stars): 10 participants (41.7%)
 +
* Lower confidence (1-2 stars): 2 participants (8.4%)
 +
* '''Average expertise weight:''' 3.67/5.0
 +
 +
'''Professional Background:'''
 +
* Music researchers: 12 (54.5%)
 +
* Music technologists: 10 (45.5%)
 +
* Active performers: 8 (36.4%)
 +
* Conservatory students: 6 (27.3%)
 +
* Music lovers: 15 (68.2%)
 +
* Concert-goers: 8 (36.4%)
 +
 +
'''Musical Experience:'''
 +
* Strong representation of classical music expertise
 +
* Diverse musical preferences spanning classical, jazz, pop, and rock
 +
* Substantial piano experience among evaluators
 +
* Mix of academic researchers and practicing musicians
 +
 +
=== System Rankings ===
 +
 +
The following table shows the final rankings based on weighted average scores from the preliminary round evaluation:
 +
{| class="wikitable sortable"
 +
! Rank
 +
! Anonymous Name
 +
! Real System Name
 +
! Authors/Institution
 +
! Weighted Score
 +
! PDF
 +
|-
 +
| 1
 +
| MidnightOpal
 +
| DirectorMusices
 +
| Anders Friberg, Gabriel Jones
 +
| 4.33/5.0
 +
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/DirectorMusices.pdf PDF]
 +
|-
 +
| 2
 +
| CrystalEcho
 +
| VirtuosoNet
 +
| Dasaem Jeong, Taegyun Kwon, Yoojin Kim, Kyogu Lee, Juhan Nam
 +
| 3.54/5.0
 +
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/VirtuosoNet.pdf PDF]
 +
|-
 +
| 3
 +
| FrozenRiver
 +
| Midihum
 +
| Erich Grunewald
 +
| 3.32/5.0
 +
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/Midihum.pdf PDF]
 +
|-
 +
| 4
 +
| VelvetStorm
 +
| ElegantAIPianist
 +
| Leduo Chen, Xinrui Su, Yuqiang Li, Honyu Andy Shing, Junchuan Zhao, Zihan Chai, Kunyang Zhang, Shengchen Li
 +
| 3.19/5.0
 +
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/ElegantAIPianist.pdf PDF]
 +
|-
 +
| 5
 +
| SilverWave
 +
| Contin-U
 +
| Jongmin Jung, Dongmin Kim, Sihun Lee, Seola Cho, Hyungjoon Soh, Irmak Bukey, Chris Donahue, Dasaem Jeong
 +
| 3.00/5.0
 +
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/Contin-U.pdf PDF]
 +
|-
 +
| 6
 +
| EmberSky
 +
| YQX+
 +
| Jinwen Zhou, Yuncong Xie, Haochen Wang, Huan Zhang, Aidan Hogg, Simon Dixon
 +
| 2.83/5.0
 +
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/YQX+.pdf PDF]
 +
|-
 +
|
 +
| CrimsonDawn
 +
| ScorePerLockNAR
 +
| Weixi Zhai
 +
| 2.53/5.0
 +
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/ScorePerLockNAR.pdf PDF]
 +
|-
 +
|
 +
| AzureThunder
 +
| RenConnoisseur
 +
| Silvan Peter
 +
| 2.53/5.0
 +
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/RenConnoisseur.pdf PDF]
 +
|-
 +
|
 +
| GoldenMist
 +
| CueFreeExpressPedal
 +
| Kyle Worrall, Tom Collins
 +
| 2.31/5.0
 +
| [https://futuremirex.com/portal/wp-content/uploads/2025/rencon/CueFreeExpressPedal.pdf PDF]
 +
|}
 +
 +
''Note: Complete rankings and system details will be updated following the live contest and final results announcement.''
 +
 +
=== Qualitative Feedback ===
 +
 +
Evaluators provided extensive qualitative feedback on the systems' performances:
 +
 +
'''Common Positive Attributes:'''
 +
* Natural expressiveness and human-like phrasing
 +
* Appropriate tempo variations and rubato
 +
* Musical sensitivity to harmonic structure
 +
* Dynamic expression and articulation
 +
 +
'''Areas for Improvement:'''
 +
* Consistency across different musical styles
 +
* Handling of complex rhythmic patterns
 +
* Balance between technical accuracy and musical expression
 +
 +
 +
== Live Contest Results ==
 +
 +
The live contest was held on September 25, 2025, at the Daejeon Convention Center during ISMIR 2025. All systems rendered a surprise piece in real-time, with audience voting determining the final rankings. A total of '''50 evaluators''' participated in the live evaluation.
 +
 +
=== Surprise Piece ===
 +
* '''Title:''' Variation on "Mother and Sister"
 +
* '''Composer:''' Hayeon Bang
 +
* '''Duration:''' ~3 minutes
 +
 +
 +
=== Live Contest Rankings ===
 +
 +
{| class="wikitable sortable"
 +
! Rank
 +
! System Index
 +
! Model Name
 +
! Authors
 +
! Live Score
 +
! Preliminary Rank
 +
! Score Change
 +
|-
 +
| 1st
 +
| #8
 +
| VirtuosoNet
 +
| Dasaem Jeong et al.
 +
| 3.62/5.0
 +
| 2nd
 +
| ↑1
 +
|-
 +
| 2nd
 +
| #1
 +
| DirectorMusices
 +
| Anders Friberg et al.
 +
| 3.06/5.0
 +
| 1st
 +
| ↓1
 +
|-
 +
| 3rd
 +
| #2
 +
| Midihum
 +
| Erich Grunewald
 +
| 2.90/5.0
 +
| 3rd
 +
| —
 +
|-
 +
| 4th
 +
| #4
 +
| Contin-U
 +
| Jongmin Jung et al.
 +
| 2.90/5.0
 +
| 5th
 +
| ↑1
 +
|-
 +
| 5th
 +
| #6
 +
| ScorePerLockNAR
 +
| Weixi Zhai
 +
| 2.52/5.0
 +
| 7th
 +
| ↑2
 +
|-
 +
| 6th
 +
| #3
 +
| RenConnoisseur
 +
| Silvan Peter
 +
| 2.40/5.0
 +
| 8th
 +
| ↑2
 +
|-
 +
| 7th
 +
| #9
 +
| ElegantAIPianist
 +
| Leduo Chen et al.
 +
| 2.08/5.0
 +
| 4th
 +
| ↓3
 +
|-
 +
| 8th
 +
| #5
 +
| YQX+
 +
| Jinwen Zhou et al.
 +
| 1.79/5.0
 +
| 6th
 +
| ↓2
 +
|}
 +
 +
=== Human Baseline Performance ===
 +
A human performance of the same surprise piece was also evaluated by the audience as a reference point:
 +
 +
{| class="wikitable"
 +
! Performance Type
 +
! Score
 +
! Rank Among All Performances
 +
|-
 +
| Human Performance
 +
| 4.40/5.0
 +
| 1st (Overall)
 +
|}
 +
 +
=== Performance Analysis ===
 +
 +
'''Top Performers:'''
 +
* '''VirtuosoNet''' achieved the highest AI system score (3.62/5.0), improving from 2nd place in the preliminary round
 +
* '''DirectorMusices''' maintained strong performance (3.06/5.0) but dropped from 1st to 2nd place
 +
* '''Midihum''' showed consistency, maintaining 3rd place across both rounds
 +
 +
'''Notable Changes:'''
 +
* '''ScorePerLockNAR''' and '''RenConnoisseur''' both improved by 2 positions in the live contest
 +
* '''ElegantAIPianist''' experienced the largest drop, falling from 4th to 7th place
 +
* '''Contin-U''' improved from 5th to 4th place in live evaluation
 +
 +
'''Human vs. AI Performance:'''
 +
* The human performance scored 4.40/5.0, significantly higher than the top AI system
 +
* This represents a performance gap of 0.78 points between human and best AI performance
 +
* All AI systems scored below the human baseline, indicating continued room for improvement
 +
 +
=== Final Winner ===
 +
 +
'''🏆 RenCon 2025 Winner: VirtuosoNet'''
 +
* '''Team:''' Dasaem Jeong, Taegyun Kwon, Yoojin Kim, Kyogu Lee, Juhan Nam
 +
* '''Final Score:''' 3.62/5.0
 +
* '''Achievement:''' Highest-scoring AI system in live contest evaluation
 +
 +
The winner was announced at the closing ceremony of ISMIR 2025 on September 25, 2025.
 +
 +
 +
 +
 +
== External Links ==
 +
 +
* [https://ren-con2025.vercel.app/ Official RenCon 2025 Website]
 +
* [https://ismir2025.ismir.net/ ISMIR 2025 Conference Website]
 +
* [https://www.music-ir.org/mirex/wiki/2025:RenCon RenCon 2025 MIREX Task Page]
 +
 +
[[Category:MIREX]]
 +
[[Category:ISMIR 2025]]
 +
[[Category:Performance Rendering]]
 +
[[Category:Competition Results]]

Latest revision as of 10:22, 27 September 2025

2025:RenCon Results

Preliminary (Audition) Round Results

Evaluation Methodology

The preliminary round was evaluated through an online listening test with 25 expert evaluators. The evaluation used a weighted voting system where participants self-rated their expertise level from 1-5 stars, with responses weighted accordingly.

Participant Demographics

Our evaluation panel consisted of highly qualified judges:

Expertise Distribution:

  • Expert evaluators (5 stars): 7 participants (29.2%)
  • High confidence (4 stars): 5 participants (20.8%)
  • Moderate confidence (3 stars): 10 participants (41.7%)
  • Lower confidence (1-2 stars): 2 participants (8.4%)
  • Average expertise weight: 3.67/5.0

Professional Background:

  • Music researchers: 12 (54.5%)
  • Music technologists: 10 (45.5%)
  • Active performers: 8 (36.4%)
  • Conservatory students: 6 (27.3%)
  • Music lovers: 15 (68.2%)
  • Concert-goers: 8 (36.4%)

Musical Experience:

  • Strong representation of classical music expertise
  • Diverse musical preferences spanning classical, jazz, pop, and rock
  • Substantial piano experience among evaluators
  • Mix of academic researchers and practicing musicians

System Rankings

The following table shows the final rankings based on weighted average scores from the preliminary round evaluation:

Rank Anonymous Name Real System Name Authors/Institution Weighted Score PDF
1 MidnightOpal DirectorMusices Anders Friberg, Gabriel Jones 4.33/5.0 PDF
2 CrystalEcho VirtuosoNet Dasaem Jeong, Taegyun Kwon, Yoojin Kim, Kyogu Lee, Juhan Nam 3.54/5.0 PDF
3 FrozenRiver Midihum Erich Grunewald 3.32/5.0 PDF
4 VelvetStorm ElegantAIPianist Leduo Chen, Xinrui Su, Yuqiang Li, Honyu Andy Shing, Junchuan Zhao, Zihan Chai, Kunyang Zhang, Shengchen Li 3.19/5.0 PDF
5 SilverWave Contin-U Jongmin Jung, Dongmin Kim, Sihun Lee, Seola Cho, Hyungjoon Soh, Irmak Bukey, Chris Donahue, Dasaem Jeong 3.00/5.0 PDF
6 EmberSky YQX+ Jinwen Zhou, Yuncong Xie, Haochen Wang, Huan Zhang, Aidan Hogg, Simon Dixon 2.83/5.0 PDF
CrimsonDawn ScorePerLockNAR Weixi Zhai 2.53/5.0 PDF
AzureThunder RenConnoisseur Silvan Peter 2.53/5.0 PDF
GoldenMist CueFreeExpressPedal Kyle Worrall, Tom Collins 2.31/5.0 PDF

Note: Complete rankings and system details will be updated following the live contest and final results announcement.

Qualitative Feedback

Evaluators provided extensive qualitative feedback on the systems' performances:

Common Positive Attributes:

  • Natural expressiveness and human-like phrasing
  • Appropriate tempo variations and rubato
  • Musical sensitivity to harmonic structure
  • Dynamic expression and articulation

Areas for Improvement:

  • Consistency across different musical styles
  • Handling of complex rhythmic patterns
  • Balance between technical accuracy and musical expression


Live Contest Results

The live contest was held on September 25, 2025, at the Daejeon Convention Center during ISMIR 2025. All systems rendered a surprise piece in real-time, with audience voting determining the final rankings. A total of 50 evaluators participated in the live evaluation.

Surprise Piece

  • Title: Variation on "Mother and Sister"
  • Composer: Hayeon Bang
  • Duration: ~3 minutes


Live Contest Rankings

Rank System Index Model Name Authors Live Score Preliminary Rank Score Change
1st #8 VirtuosoNet Dasaem Jeong et al. 3.62/5.0 2nd ↑1
2nd #1 DirectorMusices Anders Friberg et al. 3.06/5.0 1st ↓1
3rd #2 Midihum Erich Grunewald 2.90/5.0 3rd —
4th #4 Contin-U Jongmin Jung et al. 2.90/5.0 5th ↑1
5th #6 ScorePerLockNAR Weixi Zhai 2.52/5.0 7th ↑2
6th #3 RenConnoisseur Silvan Peter 2.40/5.0 8th ↑2
7th #9 ElegantAIPianist Leduo Chen et al. 2.08/5.0 4th ↓3
8th #5 YQX+ Jinwen Zhou et al. 1.79/5.0 6th ↓2

Human Baseline Performance

A human performance of the same surprise piece was also evaluated by the audience as a reference point:

Performance Type Score Rank Among All Performances
Human Performance 4.40/5.0 1st (Overall)

Performance Analysis

Top Performers:

  • VirtuosoNet achieved the highest AI system score (3.62/5.0), improving from 2nd place in the preliminary round
  • DirectorMusices maintained strong performance (3.06/5.0) but dropped from 1st to 2nd place
  • Midihum showed consistency, maintaining 3rd place across both rounds

Notable Changes:

  • ScorePerLockNAR and RenConnoisseur both improved by 2 positions in the live contest
  • ElegantAIPianist experienced the largest drop, falling from 4th to 7th place
  • Contin-U improved from 5th to 4th place in live evaluation

Human vs. AI Performance:

  • The human performance scored 4.40/5.0, significantly higher than the top AI system
  • This represents a performance gap of 0.78 points between human and best AI performance
  • All AI systems scored below the human baseline, indicating continued room for improvement

Final Winner

🏆 RenCon 2025 Winner: VirtuosoNet

  • Team: Dasaem Jeong, Taegyun Kwon, Yoojin Kim, Kyogu Lee, Juhan Nam
  • Final Score: 3.62/5.0
  • Achievement: Highest-scoring AI system in live contest evaluation

The winner was announced at the closing ceremony of ISMIR 2025 on September 25, 2025.



External Links

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2025:RenCon_Results&oldid=14841"