Difference between revisions of "2021: Automatic Lyrics Transcription Results"
From MIREX Wiki
(7 intermediate revisions by the same user not shown) | |||
Line 10: | Line 10: | ||
! YYHL1 | ! YYHL1 | ||
− | | NetEase || style="text-align: center;" | [https:// | + | | NetEase || style="text-align: center;" | [https://drive.google.com/file/d/1yxwIZcQTvyb_IgN37lxQuy4TFKAJHwrZ/view?usp=sharing PDF] || Zhen Yang, Qichen Han, Xiang Li, Dong Liu, Peng Li |
|- | |- | ||
! YYHL3 | ! YYHL3 | ||
− | | NetEase || style="text-align: center;" | [https:// | + | | NetEase || style="text-align: center;" | [https://drive.google.com/file/d/1yxwIZcQTvyb_IgN37lxQuy4TFKAJHwrZ/view?usp=sharing PDF] || Zhen Yang, Qichen Han, Xiang Li, Dong Liu, Peng Li |
|- | |- | ||
|} | |} | ||
− | + | NOTE: Other submissions have not been able to reach results, because their algorithms exceeded the time and hardware limits described in the task rules. | |
=Results= | =Results= | ||
Line 43: | Line 43: | ||
====Per-track results==== | ====Per-track results==== | ||
+ | YYHL1 | ||
+ | <csv>2021/lt/hansen_solo_yhll1.csv</csv> | ||
+ | YYHL3 | ||
+ | <csv>2021/lt/hansen_solo_yhll3.csv</csv> | ||
===Hansen's dataset=== | ===Hansen's dataset=== | ||
Line 63: | Line 67: | ||
====Per-track results==== | ====Per-track results==== | ||
+ | YYHL1 | ||
+ | <csv>2021/lt/hansen_poli_yhll1.csv</csv> | ||
+ | |||
+ | YYHL3 | ||
+ | <csv>2021/lt/hansen_poli_yhll3.csv</csv> | ||
===Mauch's dataset=== | ===Mauch's dataset=== | ||
Line 82: | Line 91: | ||
|} | |} | ||
====Per-track results==== | ====Per-track results==== | ||
+ | |||
+ | YYHL1 | ||
+ | <csv>2021/lt/mauch_yhll1.csv</csv> | ||
+ | |||
+ | YYHL3 | ||
+ | <csv>2021/lt/mauch_yhll3.csv</csv> | ||
===Jamendo dataset=== | ===Jamendo dataset=== | ||
Line 101: | Line 116: | ||
|} | |} | ||
====Per-track results==== | ====Per-track results==== | ||
+ | |||
+ | YYHL1 | ||
+ | <csv>2021/lt/jamendo_yhll1.csv</csv> | ||
+ | |||
+ | YYHL3 | ||
+ | <csv>2021/lt/jamendo_yhll3.csv</csv> |
Latest revision as of 12:14, 20 March 2022
Contents
General Legend
Sub code | Submission name | Abstract | Contributors |
---|---|---|---|
YYHL1 | NetEase | Zhen Yang, Qichen Han, Xiang Li, Dong Liu, Peng Li | |
YYHL3 | NetEase | Zhen Yang, Qichen Han, Xiang Li, Dong Liu, Peng Li |
NOTE: Other submissions have not been able to reach results, because their algorithms exceeded the time and hardware limits described in the task rules.
Results
Hansen's dataset a cappella
Summary Results
Submission | Avrg WER |
---|---|
YYHL1 | 11.45 |
YYHL3 | 12.77 |
Per-track results
YYHL1
ID | WER | C | S | I | D | Total_words |
---|---|---|---|---|---|---|
rolling_in_the_deep | 0.08611111111111111 | 333 | 17 | 4 | 10 | 360 |
rehab | 0.08823529411764706 | 289 | 16 | 10 | 1 | 306 |
Beautiful_Stranger_full_ending | 0.21739130434782608 | 234 | 17 | 0 | 48 | 299 |
sunrise | 0.312 | 87 | 14 | 1 | 24 | 125 |
umbrella | 0.18491484184914841 | 335 | 38 | 0 | 38 | 411 |
trick_me | 0.08620689655172414 | 267 | 17 | 2 | 6 | 290 |
i_kissed_a_girl | 0.007575757575757576 | 263 | 1 | 1 | 0 | 264 |
clocks_cut | 0.09411764705882353 | 78 | 5 | 1 | 2 | 85 |
viva_la_vida | 0.05673758865248227 | 267 | 15 | 1 | 0 | 282 |
this_afternoon | 0.08849557522123894 | 423 | 19 | 11 | 10 | 452 |
YYHL3
ID | WER | C | S | I | D | Total_words |
---|---|---|---|---|---|---|
rolling_in_the_deep | 0.09166666666666666 | 332 | 20 | 5 | 8 | 360 |
rehab | 0.06862745098039216 | 294 | 7 | 9 | 5 | 306 |
Beautiful_Stranger_full_ending | 0.22073578595317725 | 233 | 22 | 0 | 44 | 299 |
sunrise | 0.344 | 84 | 16 | 2 | 25 | 125 |
umbrella | 0.1897810218978102 | 335 | 21 | 2 | 55 | 411 |
trick_me | 0.08620689655172414 | 272 | 9 | 7 | 9 | 290 |
i_kissed_a_girl | 0.03409090909090909 | 260 | 0 | 5 | 4 | 264 |
clocks_cut | 0.17647058823529413 | 70 | 9 | 0 | 6 | 85 |
viva_la_vida | 0.09929078014184398 | 254 | 23 | 0 | 5 | 282 |
this_afternoon | 0.1084070796460177 | 412 | 31 | 9 | 9 | 452 |
Hansen's dataset
Summary Results
Submission | Avrg WER |
---|---|
YYHL1 | 13.54 |
YYHL3 | 16.88 |
Per-track results
YYHL1
ID | WER | C | S | I | D | Total_words |
---|---|---|---|---|---|---|
rolling_in_the_deep | 0.08611111111111111 | 332 | 14 | 3 | 14 | 360 |
rehab | 0.10130718954248366 | 286 | 15 | 11 | 5 | 306 |
Beautiful_Stranger_full_ending | 0.23076923076923078 | 230 | 16 | 0 | 53 | 299 |
sunrise | 0.32 | 86 | 14 | 1 | 25 | 125 |
umbrella | 0.20924574209245742 | 328 | 38 | 3 | 45 | 411 |
trick_me | 0.13793103448275862 | 260 | 19 | 10 | 11 | 290 |
i_kissed_a_girl | 0.007575757575757576 | 263 | 1 | 1 | 0 | 264 |
clocks_cut | 0.12941176470588237 | 75 | 7 | 1 | 3 | 85 |
viva_la_vida | 0.06382978723404255 | 266 | 16 | 2 | 0 | 282 |
this_afternoon | 0.13495575221238937 | 417 | 24 | 26 | 11 | 452 |
YYHL3
ID | WER | C | S | I | D | Total_words |
---|---|---|---|---|---|---|
rolling_in_the_deep | 0.09444444444444444 | 330 | 17 | 4 | 13 | 360 |
rehab | 0.13398692810457516 | 272 | 14 | 7 | 20 | 306 |
Beautiful_Stranger_full_ending | 0.2408026755852843 | 229 | 16 | 2 | 54 | 299 |
sunrise | 0.384 | 78 | 16 | 1 | 31 | 125 |
umbrella | 0.2798053527980535 | 299 | 28 | 3 | 84 | 411 |
trick_me | 0.12758620689655173 | 265 | 12 | 12 | 13 | 290 |
i_kissed_a_girl | 0.041666666666666664 | 257 | 0 | 4 | 7 | 264 |
clocks_cut | 0.12941176470588237 | 74 | 6 | 0 | 5 | 85 |
viva_la_vida | 0.1524822695035461 | 241 | 33 | 2 | 8 | 282 |
this_afternoon | 0.16150442477876106 | 406 | 33 | 27 | 13 | 452 |
Mauch's dataset
Summary Results
Submission | Avrg WER |
---|---|
YYHL1 | 22.11 |
YYHL3 | 26.20 |
Per-track results
YYHL1
ID | WER | C | S | I | D | Total_words |
---|---|---|---|---|---|---|
Abba.KnowingMeKnowingYou | 0.15979381443298968 | 179 | 3 | 16 | 12 | 194 |
Bangles.EternalFlame | 0.1607843137254902 | 216 | 2 | 2 | 37 | 255 |
Blondie.CallMe | 0.3480392156862745 | 133 | 1 | 0 | 70 | 204 |
Duffy.WarwickAvenue | 0.11406844106463879 | 235 | 23 | 2 | 5 | 263 |
DuranDuran.OrdinaryWorld | 0.12307692307692308 | 229 | 6 | 1 | 25 | 260 |
FranzFerdinand.DoYouWantTo | 0.3855721393034826 | 251 | 21 | 4 | 130 | 402 |
Martika.ToySoldiers | 0.6541666666666667 | 87 | 14 | 4 | 139 | 240 |
Muse.GuidingLight | 0.15517241379310345 | 98 | 1 | 0 | 17 | 116 |
OtisRedding.TheDockOfTheBay | 0.1783783783783784 | 154 | 24 | 2 | 7 | 185 |
Queen.WeAreTheChampions | 0.1945945945945946 | 150 | 8 | 1 | 27 | 185 |
RM.P082 | 0.24615384615384617 | 245 | 36 | 0 | 44 | 325 |
RM.P084 | 0.3890577507598784 | 202 | 62 | 1 | 65 | 329 |
RobertPalmer.AddictedToLove | 0.1235632183908046 | 312 | 10 | 7 | 26 | 348 |
Santana.BlackMagicWoman | 0.075 | 111 | 4 | 0 | 5 | 120 |
SimonAndGarfunkel.Cecilia | 0.48868778280542985 | 113 | 14 | 0 | 94 | 221 |
TakeThat.BackForGood | 0.15303430079155672 | 325 | 10 | 4 | 44 | 379 |
TinaTurner.WhatsLoveGotToDoWithIt | 0.05263157894736842 | 307 | 0 | 1 | 16 | 323 |
Toto.Africa | 0.07196969696969698 | 255 | 9 | 10 | 0 | 264 |
U2.WithOrWithoutYou | 0.1099476439790576 | 170 | 1 | 0 | 20 | 191 |
Zweieck.She | 0.11255411255411256 | 207 | 6 | 2 | 18 | 231 |
YYHL3
ID | WER | C | S | I | D | Total_words |
---|---|---|---|---|---|---|
Abba.KnowingMeKnowingYou | 0.10309278350515463 | 179 | 5 | 5 | 10 | 194 |
Bangles.EternalFlame | 0.1803921568627451 | 211 | 2 | 2 | 42 | 255 |
Blondie.CallMe | 0.5882352941176471 | 84 | 18 | 0 | 102 | 204 |
Duffy.WarwickAvenue | 0.053231939163498096 | 251 | 6 | 2 | 6 | 263 |
DuranDuran.OrdinaryWorld | 0.17692307692307693 | 216 | 13 | 2 | 31 | 260 |
FranzFerdinand.DoYouWantTo | 0.39303482587064675 | 246 | 18 | 2 | 138 | 402 |
Martika.ToySoldiers | 0.7 | 75 | 17 | 3 | 148 | 240 |
Muse.GuidingLight | 0.1724137931034483 | 96 | 5 | 0 | 15 | 116 |
OtisRedding.TheDockOfTheBay | 0.12972972972972974 | 162 | 9 | 1 | 14 | 185 |
Queen.WeAreTheChampions | 0.2864864864864865 | 132 | 20 | 0 | 33 | 185 |
RM.P082 | 0.23384615384615384 | 249 | 42 | 0 | 34 | 325 |
RM.P084 | 0.3951367781155015 | 201 | 73 | 2 | 55 | 329 |
RobertPalmer.AddictedToLove | 0.2614942528735632 | 264 | 26 | 7 | 58 | 348 |
Santana.BlackMagicWoman | 0.125 | 105 | 8 | 0 | 7 | 120 |
SimonAndGarfunkel.Cecilia | 0.5339366515837104 | 103 | 3 | 0 | 115 | 221 |
TakeThat.BackForGood | 0.25065963060686014 | 286 | 32 | 2 | 61 | 379 |
TinaTurner.WhatsLoveGotToDoWithIt | 0.19814241486068113 | 265 | 13 | 6 | 45 | 323 |
Toto.Africa | 0.022727272727272728 | 260 | 4 | 2 | 0 | 264 |
U2.WithOrWithoutYou | 0.1099476439790576 | 170 | 3 | 0 | 18 | 191 |
Zweieck.She | 0.1471861471861472 | 199 | 12 | 2 | 20 | 231 |
Jamendo dataset
Summary Results
Submission | Avrg WER |
---|---|
YYHL1 | 24.34 |
YYHL3 | 26.86 |
Per-track results
YYHL1
ID | WER | C | S | I | D | Total_words |
---|---|---|---|---|---|---|
Avercage_-_Embers | 0.7407407407407407 | 49 | 8 | 0 | 132 | 189 |
Color_Out_-_Falling_Star | 0.16964285714285715 | 191 | 21 | 5 | 12 | 224 |
Cortez_-_Feel__Stripped_ | 0.11048158640226628 | 320 | 20 | 6 | 13 | 353 |
Explosive_Ear_Candy_-_Like_The_Sun | 0.09056603773584905 | 241 | 19 | 0 | 5 | 265 |
HILA_-_Give_Me_the_Same | 0.14906832298136646 | 275 | 16 | 1 | 31 | 322 |
JASON_MILLER_-_CROWD_PLEASER | 0.2591170825335892 | 399 | 107 | 13 | 15 | 521 |
Kinematic_-_Peyote | 0.25170068027210885 | 112 | 20 | 2 | 15 | 147 |
LUNABLIND_-_Vision__Radio_Edit_ | 0.08680555555555555 | 270 | 12 | 7 | 6 | 288 |
Lower_Loveday_-_Is_It_Right_ | 0.12735849056603774 | 187 | 13 | 2 | 12 | 212 |
Moon_I_Mean_-_Wrong_Concept | 0.602996254681648 | 106 | 40 | 0 | 121 | 267 |
Pure_Mids_-_The_Leader | 0.5350877192982456 | 54 | 8 | 1 | 52 | 114 |
Quentin_Hannappe_-_Keep_On | 0.21714285714285714 | 140 | 22 | 3 | 13 | 175 |
Ridgway_-_Fire_Inside | 0.4506578947368421 | 175 | 40 | 8 | 89 | 304 |
Rxbyn_-_Bad_Side | 0.18409090909090908 | 365 | 15 | 6 | 60 | 440 |
Slingshot_Miracle_-_Whistler | 0.2345679012345679 | 124 | 12 | 0 | 26 | 162 |
Songwriterz_-_Back_In_Time | 0.2542372881355932 | 177 | 28 | 1 | 31 | 236 |
The.madpix.project_-_One_Way_Street | 0.20218579234972678 | 147 | 32 | 1 | 4 | 183 |
The_Rinn_-_Voices__2017_Version_ | 0.30198019801980197 | 144 | 21 | 3 | 37 | 202 |
Tom_Orlando_-_The_One__feat._Tina_G_ | 0.20967741935483872 | 396 | 52 | 4 | 48 | 496 |
Wordsmith_-_The_Statement | 0.15834767641996558 | 504 | 57 | 15 | 20 | 581 |
YYHL3
ID | WER | C | S | I | D | Total_words |
---|---|---|---|---|---|---|
Avercage_-_Embers | 0.708994708994709 | 55 | 12 | 0 | 122 | 189 |
Color_Out_-_Falling_Star | 0.17857142857142858 | 190 | 23 | 6 | 11 | 224 |
Cortez_-_Feel__Stripped_ | 0.1388101983002833 | 315 | 15 | 11 | 23 | 353 |
Explosive_Ear_Candy_-_Like_The_Sun | 0.11320754716981132 | 235 | 24 | 0 | 6 | 265 |
HILA_-_Give_Me_the_Same | 0.16149068322981366 | 272 | 17 | 2 | 33 | 322 |
JASON_MILLER_-_CROWD_PLEASER | 0.2418426103646833 | 407 | 93 | 12 | 21 | 521 |
Kinematic_-_Peyote | 0.2857142857142857 | 106 | 23 | 1 | 18 | 147 |
LUNABLIND_-_Vision__Radio_Edit_ | 0.12152777777777778 | 259 | 17 | 6 | 12 | 288 |
Lower_Loveday_-_Is_It_Right_ | 0.14150943396226415 | 186 | 15 | 4 | 11 | 212 |
Moon_I_Mean_-_Wrong_Concept | 0.6067415730337079 | 106 | 39 | 1 | 122 | 267 |
Pure_Mids_-_The_Leader | 0.5087719298245614 | 57 | 9 | 1 | 48 | 114 |
Quentin_Hannappe_-_Keep_On | 0.17714285714285713 | 145 | 23 | 1 | 7 | 175 |
Ridgway_-_Fire_Inside | 0.46381578947368424 | 169 | 47 | 6 | 88 | 304 |
Rxbyn_-_Bad_Side | 0.18181818181818182 | 363 | 28 | 3 | 49 | 440 |
Slingshot_Miracle_-_Whistler | 0.2716049382716049 | 120 | 19 | 2 | 23 | 162 |
Songwriterz_-_Back_In_Time | 0.2796610169491525 | 172 | 28 | 2 | 36 | 236 |
The.madpix.project_-_One_Way_Street | 0.22950819672131148 | 142 | 35 | 1 | 6 | 183 |
The_Rinn_-_Voices__2017_Version_ | 0.3910891089108911 | 126 | 33 | 3 | 43 | 202 |
Tom_Orlando_-_The_One__feat._Tina_G_ | 0.3084677419354839 | 349 | 83 | 6 | 64 | 496 |
Wordsmith_-_The_Statement | 0.22719449225473323 | 462 | 103 | 13 | 16 | 581 |