Difference between revisions of "2021: Automatic Lyrics Transcription Results"

From MIREX Wiki
 
(4 intermediate revisions by the same user not shown)
Line 10: Line 10:
  
 
     ! YYHL1
 
     ! YYHL1
−
     | NetEase ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2021/report.pdf PDF] || Zhen Yang, Qichen Han, Xiang Li, Dong Liu, Peng Li
+
     | NetEase ||  style="text-align: center;" | [https://drive.google.com/file/d/1yxwIZcQTvyb_IgN37lxQuy4TFKAJHwrZ/view?usp=sharing PDF] || Zhen Yang, Qichen Han, Xiang Li, Dong Liu, Peng Li
 
     |-
 
     |-
  
 
     ! YYHL3
 
     ! YYHL3
−
     | NetEase ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2021/YYHL3.pdf PDF] || Zhen Yang, Qichen Han, Xiang Li, Dong Liu, Peng Li
+
     | NetEase ||  style="text-align: center;" | [https://drive.google.com/file/d/1yxwIZcQTvyb_IgN37lxQuy4TFKAJHwrZ/view?usp=sharing PDF] || Zhen Yang, Qichen Han, Xiang Li, Dong Liu, Peng Li
 
     |-
 
     |-
  
  
 
|}
 
|}
−
 
+
NOTE: Other submissions have not been able to reach results, because their algorithms exceeded the time and hardware limits described in the task rules.
  
 
=Results=
 
=Results=
Line 45: Line 45:
 
YYHL1
 
YYHL1
 
<csv>2021/lt/hansen_solo_yhll1.csv</csv>
 
<csv>2021/lt/hansen_solo_yhll1.csv</csv>
 +
 
YYHL3
 
YYHL3
 
<csv>2021/lt/hansen_solo_yhll3.csv</csv>
 
<csv>2021/lt/hansen_solo_yhll3.csv</csv>
Line 68: Line 69:
 
YYHL1
 
YYHL1
 
<csv>2021/lt/hansen_poli_yhll1.csv</csv>
 
<csv>2021/lt/hansen_poli_yhll1.csv</csv>
 +
 
YYHL3
 
YYHL3
 
<csv>2021/lt/hansen_poli_yhll3.csv</csv>
 
<csv>2021/lt/hansen_poli_yhll3.csv</csv>
Line 92: Line 94:
 
YYHL1
 
YYHL1
 
<csv>2021/lt/mauch_yhll1.csv</csv>
 
<csv>2021/lt/mauch_yhll1.csv</csv>
 +
 
YYHL3
 
YYHL3
 
<csv>2021/lt/mauch_yhll3.csv</csv>
 
<csv>2021/lt/mauch_yhll3.csv</csv>
Line 113: Line 116:
 
|}
 
|}
 
====Per-track results====
 
====Per-track results====
 +
 
YYHL1
 
YYHL1
 
<csv>2021/lt/jamendo_yhll1.csv</csv>
 
<csv>2021/lt/jamendo_yhll1.csv</csv>
 +
 
YYHL3
 
YYHL3
 
<csv>2021/lt/jamendo_yhll3.csv</csv>
 
<csv>2021/lt/jamendo_yhll3.csv</csv>

Latest revision as of 12:14, 20 March 2022

General Legend

Sub code Submission name Abstract Contributors
YYHL1 NetEase PDF Zhen Yang, Qichen Han, Xiang Li, Dong Liu, Peng Li
YYHL3 NetEase PDF Zhen Yang, Qichen Han, Xiang Li, Dong Liu, Peng Li

NOTE: Other submissions have not been able to reach results, because their algorithms exceeded the time and hardware limits described in the task rules.

Results

Hansen's dataset a cappella

Summary Results

Submission Avrg WER
YYHL1 11.45
YYHL3 12.77

Per-track results

YYHL1

ID WER C S I D Total_words
rolling_in_the_deep 0.08611111111111111 333 17 4 10 360
rehab 0.08823529411764706 289 16 10 1 306
Beautiful_Stranger_full_ending 0.21739130434782608 234 17 0 48 299
sunrise 0.312 87 14 1 24 125
umbrella 0.18491484184914841 335 38 0 38 411
trick_me 0.08620689655172414 267 17 2 6 290
i_kissed_a_girl 0.007575757575757576 263 1 1 0 264
clocks_cut 0.09411764705882353 78 5 1 2 85
viva_la_vida 0.05673758865248227 267 15 1 0 282
this_afternoon 0.08849557522123894 423 19 11 10 452

download these results as csv

YYHL3

ID WER C S I D Total_words
rolling_in_the_deep 0.09166666666666666 332 20 5 8 360
rehab 0.06862745098039216 294 7 9 5 306
Beautiful_Stranger_full_ending 0.22073578595317725 233 22 0 44 299
sunrise 0.344 84 16 2 25 125
umbrella 0.1897810218978102 335 21 2 55 411
trick_me 0.08620689655172414 272 9 7 9 290
i_kissed_a_girl 0.03409090909090909 260 0 5 4 264
clocks_cut 0.17647058823529413 70 9 0 6 85
viva_la_vida 0.09929078014184398 254 23 0 5 282
this_afternoon 0.1084070796460177 412 31 9 9 452

download these results as csv

Hansen's dataset

Summary Results

Submission Avrg WER
YYHL1 13.54
YYHL3 16.88

Per-track results

YYHL1

ID WER C S I D Total_words
rolling_in_the_deep 0.08611111111111111 332 14 3 14 360
rehab 0.10130718954248366 286 15 11 5 306
Beautiful_Stranger_full_ending 0.23076923076923078 230 16 0 53 299
sunrise 0.32 86 14 1 25 125
umbrella 0.20924574209245742 328 38 3 45 411
trick_me 0.13793103448275862 260 19 10 11 290
i_kissed_a_girl 0.007575757575757576 263 1 1 0 264
clocks_cut 0.12941176470588237 75 7 1 3 85
viva_la_vida 0.06382978723404255 266 16 2 0 282
this_afternoon 0.13495575221238937 417 24 26 11 452

download these results as csv

YYHL3

ID WER C S I D Total_words
rolling_in_the_deep 0.09444444444444444 330 17 4 13 360
rehab 0.13398692810457516 272 14 7 20 306
Beautiful_Stranger_full_ending 0.2408026755852843 229 16 2 54 299
sunrise 0.384 78 16 1 31 125
umbrella 0.2798053527980535 299 28 3 84 411
trick_me 0.12758620689655173 265 12 12 13 290
i_kissed_a_girl 0.041666666666666664 257 0 4 7 264
clocks_cut 0.12941176470588237 74 6 0 5 85
viva_la_vida 0.1524822695035461 241 33 2 8 282
this_afternoon 0.16150442477876106 406 33 27 13 452

download these results as csv

Mauch's dataset

Summary Results

Submission Avrg WER
YYHL1 22.11
YYHL3 26.20

Per-track results

YYHL1

ID WER C S I D Total_words
Abba.KnowingMeKnowingYou 0.15979381443298968 179 3 16 12 194
Bangles.EternalFlame 0.1607843137254902 216 2 2 37 255
Blondie.CallMe 0.3480392156862745 133 1 0 70 204
Duffy.WarwickAvenue 0.11406844106463879 235 23 2 5 263
DuranDuran.OrdinaryWorld 0.12307692307692308 229 6 1 25 260
FranzFerdinand.DoYouWantTo 0.3855721393034826 251 21 4 130 402
Martika.ToySoldiers 0.6541666666666667 87 14 4 139 240
Muse.GuidingLight 0.15517241379310345 98 1 0 17 116
OtisRedding.TheDockOfTheBay 0.1783783783783784 154 24 2 7 185
Queen.WeAreTheChampions 0.1945945945945946 150 8 1 27 185
RM.P082 0.24615384615384617 245 36 0 44 325
RM.P084 0.3890577507598784 202 62 1 65 329
RobertPalmer.AddictedToLove 0.1235632183908046 312 10 7 26 348
Santana.BlackMagicWoman 0.075 111 4 0 5 120
SimonAndGarfunkel.Cecilia 0.48868778280542985 113 14 0 94 221
TakeThat.BackForGood 0.15303430079155672 325 10 4 44 379
TinaTurner.WhatsLoveGotToDoWithIt 0.05263157894736842 307 0 1 16 323
Toto.Africa 0.07196969696969698 255 9 10 0 264
U2.WithOrWithoutYou 0.1099476439790576 170 1 0 20 191
Zweieck.She 0.11255411255411256 207 6 2 18 231

download these results as csv

YYHL3

ID WER C S I D Total_words
Abba.KnowingMeKnowingYou 0.10309278350515463 179 5 5 10 194
Bangles.EternalFlame 0.1803921568627451 211 2 2 42 255
Blondie.CallMe 0.5882352941176471 84 18 0 102 204
Duffy.WarwickAvenue 0.053231939163498096 251 6 2 6 263
DuranDuran.OrdinaryWorld 0.17692307692307693 216 13 2 31 260
FranzFerdinand.DoYouWantTo 0.39303482587064675 246 18 2 138 402
Martika.ToySoldiers 0.7 75 17 3 148 240
Muse.GuidingLight 0.1724137931034483 96 5 0 15 116
OtisRedding.TheDockOfTheBay 0.12972972972972974 162 9 1 14 185
Queen.WeAreTheChampions 0.2864864864864865 132 20 0 33 185
RM.P082 0.23384615384615384 249 42 0 34 325
RM.P084 0.3951367781155015 201 73 2 55 329
RobertPalmer.AddictedToLove 0.2614942528735632 264 26 7 58 348
Santana.BlackMagicWoman 0.125 105 8 0 7 120
SimonAndGarfunkel.Cecilia 0.5339366515837104 103 3 0 115 221
TakeThat.BackForGood 0.25065963060686014 286 32 2 61 379
TinaTurner.WhatsLoveGotToDoWithIt 0.19814241486068113 265 13 6 45 323
Toto.Africa 0.022727272727272728 260 4 2 0 264
U2.WithOrWithoutYou 0.1099476439790576 170 3 0 18 191
Zweieck.She 0.1471861471861472 199 12 2 20 231

download these results as csv

Jamendo dataset

Summary Results

Submission Avrg WER
YYHL1 24.34
YYHL3 26.86

Per-track results

YYHL1

ID WER C S I D Total_words
Avercage_-_Embers 0.7407407407407407 49 8 0 132 189
Color_Out_-_Falling_Star 0.16964285714285715 191 21 5 12 224
Cortez_-_Feel__Stripped_ 0.11048158640226628 320 20 6 13 353
Explosive_Ear_Candy_-_Like_The_Sun 0.09056603773584905 241 19 0 5 265
HILA_-_Give_Me_the_Same 0.14906832298136646 275 16 1 31 322
JASON_MILLER_-_CROWD_PLEASER 0.2591170825335892 399 107 13 15 521
Kinematic_-_Peyote 0.25170068027210885 112 20 2 15 147
LUNABLIND_-_Vision__Radio_Edit_ 0.08680555555555555 270 12 7 6 288
Lower_Loveday_-_Is_It_Right_ 0.12735849056603774 187 13 2 12 212
Moon_I_Mean_-_Wrong_Concept 0.602996254681648 106 40 0 121 267
Pure_Mids_-_The_Leader 0.5350877192982456 54 8 1 52 114
Quentin_Hannappe_-_Keep_On 0.21714285714285714 140 22 3 13 175
Ridgway_-_Fire_Inside 0.4506578947368421 175 40 8 89 304
Rxbyn_-_Bad_Side 0.18409090909090908 365 15 6 60 440
Slingshot_Miracle_-_Whistler 0.2345679012345679 124 12 0 26 162
Songwriterz_-_Back_In_Time 0.2542372881355932 177 28 1 31 236
The.madpix.project_-_One_Way_Street 0.20218579234972678 147 32 1 4 183
The_Rinn_-_Voices__2017_Version_ 0.30198019801980197 144 21 3 37 202
Tom_Orlando_-_The_One__feat._Tina_G_ 0.20967741935483872 396 52 4 48 496
Wordsmith_-_The_Statement 0.15834767641996558 504 57 15 20 581

download these results as csv

YYHL3

ID WER C S I D Total_words
Avercage_-_Embers 0.708994708994709 55 12 0 122 189
Color_Out_-_Falling_Star 0.17857142857142858 190 23 6 11 224
Cortez_-_Feel__Stripped_ 0.1388101983002833 315 15 11 23 353
Explosive_Ear_Candy_-_Like_The_Sun 0.11320754716981132 235 24 0 6 265
HILA_-_Give_Me_the_Same 0.16149068322981366 272 17 2 33 322
JASON_MILLER_-_CROWD_PLEASER 0.2418426103646833 407 93 12 21 521
Kinematic_-_Peyote 0.2857142857142857 106 23 1 18 147
LUNABLIND_-_Vision__Radio_Edit_ 0.12152777777777778 259 17 6 12 288
Lower_Loveday_-_Is_It_Right_ 0.14150943396226415 186 15 4 11 212
Moon_I_Mean_-_Wrong_Concept 0.6067415730337079 106 39 1 122 267
Pure_Mids_-_The_Leader 0.5087719298245614 57 9 1 48 114
Quentin_Hannappe_-_Keep_On 0.17714285714285713 145 23 1 7 175
Ridgway_-_Fire_Inside 0.46381578947368424 169 47 6 88 304
Rxbyn_-_Bad_Side 0.18181818181818182 363 28 3 49 440
Slingshot_Miracle_-_Whistler 0.2716049382716049 120 19 2 23 162
Songwriterz_-_Back_In_Time 0.2796610169491525 172 28 2 36 236
The.madpix.project_-_One_Way_Street 0.22950819672131148 142 35 1 6 183
The_Rinn_-_Voices__2017_Version_ 0.3910891089108911 126 33 3 43 202
Tom_Orlando_-_The_One__feat._Tina_G_ 0.3084677419354839 349 83 6 64 496
Wordsmith_-_The_Statement 0.22719449225473323 462 103 13 16 581

download these results as csv

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2021:_Automatic_Lyrics_Transcription_Results&oldid=13575"