Junyan: /* Submissions */ PDF links (abstracts archived on futuremirex.com)

2026-10-14T15:21:47Z

‎Submissions: PDF links (abstracts archived on futuremirex.com)

Alexandre DHooge: Create results page ALT

2026-10-14T09:40:18Z

Create results page ALT

New page

==Submissions==

{| class="wikitable" style="text-align: center;"
! Team
! PDF
! Training size
! Number of parameters
! Training hours
! Inference real-time factor
|-
| Frangiadaki et al.
|
| NA
| ~132.15B
| NA
| 1.20
|-
| UFAL-PMCG-Whisper
|
| NA
| 1.55B
| NA
| 0.0386
|-
| UFAL-PMCG-Qwen-ASR
|
| NA
| 3.6B
| NA
| 0.0969
|-
| UFAL-PMCG-Whisper+Qwen
|
| NA
| 11.8B
| NA
| 0.1753
|-
| UFAL-PMCG-Gemma
|
| NA
| 11.6B
| NA
| 0.2111
|}

==Results==

The results below are aggregate scores for each dataset. Lower is better for CER, WER, and WER'; higher is better for the F-scores. The formatting-token scores are F-scores for Punctuation (P), Parentheses (B), Line Breaks (L), and Section Breaks (S).
Because all submissions were multilingual natively, we directly report complete results on the data.
===Jam-ALT===

{| class="wikitable" style="text-align: center;"
! Team
! CER
! WER
! WER'
! F-score (P)
! F-score (B)
! F-score (L)
! F-score (S)
|-
| Frangiadaki et al.(1)
| 0.0881
| 0.1470
| 0.1913
| 0.5134
| 0.4932
| 0.8026
| 0.7587
|-
| UFAL-PMCG-Whisper
| 0.1991
| 0.2860
| 0.3317
| 0.4295
| NA
| 0.7442
| 0.0000
|-
| UFAL-PMCG-Qwen-ASR
| 0.1598
| 0.2593
| 0.3190
| 0.2584
| NA
| 0.5438
| NA
|-
| UFAL-PMCG-Whisper+Qwen
| 0.1714
| 0.2622
| 0.3162
| 0.3210
| NA
| 0.6764
| 0.2661
|-
| UFAL-PMCG-Gemma
| 0.1785
| 0.2806
| 0.3301
| 0.1447
| 0.0943
| 0.8118
| 0.2431
|}

(1) The lyrics for the Spanish songs were pre-computed by the authors, because inference ran OOM on a V100-32GB.

===MUSDB-ALT===

{| class="wikitable" style="text-align: center;"
! Team
! CER
! WER
! WER'
! F-score (P)
! F-score (B)
! F-score (L)
! F-score (S)
|-
| Frangiadaki et al.(2)
| 0.1163
| 0.1617
| 0.1970
| 0.2941
| 0.3190
| 0.8378
| 0.7485
|-
| UFAL-PMCG-Whisper
| 0.2031
| 0.2740
| 0.3126
| 0.2335
| NA
| 0.7748
| 0.2159
|-
| UFAL-PMCG-Qwen-ASR
| 0.1569
| 0.2181
| 0.2697
| 0.1174
| NA
| 0.5615
| NA
|-
| UFAL-PMCG-Whisper+Qwen
| 0.1765
| 0.2362
| 0.2858
| 0.1563
| NA
| 0.6896
| 0.2430
|-
| UFAL-PMCG-Gemma
| 0.2310
| 0.3123
| 0.3520
| 0.0680
| 0.1389
| 0.8138
| 0.2331
|}

(2) The lyrics were pre-computed by the authors, because inference ran OOM on a V100-32GB.

===JamendoLyrics++===

[https://github.com/tikick/jamendolyricspp JamendoLyrics++] was used as an additional evaluation set. It contains 80 English songs that do not overlap the previous 2 datasets.
The lyrics however do not contain punctuation or capitalization, so only a reduced set of metrics is used.

{| class="wikitable" style="text-align: center;"
! Team
! CER
! WER
! F-score (L)
! F-score (S)
|-
| Frangiadiaki et al.
| 0.1263
| 0.1768
| 0.8250
| 0.7655
|-
| UFAL-PMCG-Whisper
| 0.1835
| 0.2625
| 0.7495
| 0.2500
|-
| UFAL-PMCG-Qwen-ASR
| 0.1391
| 0.2144
| 0.5562
| NA
|-
| UFAL-PMCG-Whisper+Qwen
| 0.1747
| 0.2495
| 0.6698
| 0.2765
|-
| UFAL-PMCG-Gemma
| 0.2429
| 0.3353
| 0.7667
| 0.2269
|}

@@ Line 10: / Line 10: @@
 |-
 | Frangiadaki et al.
+| [http://futuremirex.com/portal/wp-content/uploads/2026/lyrics-transcription/Frangiadaki.pdf PDF]
 | NA
 | ~132.15B
@@ Line 17: / Line 17: @@
 |-
 | UFAL-PMCG-Whisper
+| [http://futuremirex.com/portal/wp-content/uploads/2026/lyrics-transcription/UFAL-PMCG-Whisper.pdf PDF]
 | NA
 | 1.55B
@@ Line 24: / Line 24: @@
 |-
 | UFAL-PMCG-Qwen-ASR
+| [http://futuremirex.com/portal/wp-content/uploads/2026/lyrics-transcription/UFAL-PMCG-Qwen-ASR.pdf PDF]
 | NA
 | 3.6B
@@ Line 31: / Line 31: @@
 |-
 | UFAL-PMCG-Whisper+Qwen
+| [http://futuremirex.com/portal/wp-content/uploads/2026/lyrics-transcription/UFAL-PMCG-Whisper-Qwen.pdf PDF]
 | NA
 | 11.8B
@@ Line 38: / Line 38: @@
 |-
 | UFAL-PMCG-Gemma
+| [http://futuremirex.com/portal/wp-content/uploads/2026/lyrics-transcription/UFAL-PMCG-Gemma.pdf PDF]
 | NA
 | 11.6B

2026:Lyrics Transcription Results - Revision history

Junyan: /* Submissions */ PDF links (abstracts archived on futuremirex.com)

Alexandre DHooge: Create results page ALT