Difference between revisions of "2021: Automatic Lyrics Transcription Results"

From MIREX Wiki
(→‎General Legend)
Line 19: Line 19:
  
 
|}
 
|}
 +
 +
 +
=Results=
 +
 +
===Hansen's dataset a cappella===
 +
====Summary Results====
 +
 +
| border="1" cellspacing="0" style="text-align: left; width: 800px;"
 +
    |- style="background: white"
 +
    ! width="200" | Submission
 +
    ! width="100" | Avrg WER
 +
    |-
 +
    ! RB1| Sheffield University
 +
    |-
 +
====Per-track results====
 +
 +
GGL1
 +
<csv>2020/lt/HansensDataset/YYHL1/results.csv</csv>
 +
 +
GGL2
 +
<csv>2020/lt/HansensDataset/YYHL3/results.csv</csv>
 +
 +
RB1
 +
<csv>2020/lt/HansensDataset/RB1//results.csv</csv>
 +
 +
DDA2
 +
<csv>2020/lt/HansensDataset/DDA2//results.csv</csv>
 +
 +
DDA3
 +
<csv>2020/lt/HansensDataset/DDA3//results.csv</csv>
 +
 +
 +
===Hansen's dataset===
 +
====Summary Results====
 +
<csv>2020/lt/HansensDataset/summary_HansensDataset.csv</csv>
 +
 +
====Per-track results====
 +
 +
GGL1
 +
<csv>2020/lt/HansensDataset/GGL1//results.csv</csv>
 +
 +
GGL2
 +
<csv>2020/lt/HansensDataset/GGL2//results.csv</csv>
 +
 +
RB1
 +
<csv>2020/lt/HansensDataset/RB1//results.csv</csv>
 +
 +
DDA2
 +
<csv>2020/lt/HansensDataset/DDA2//results.csv</csv>
 +
 +
DDA3
 +
<csv>2020/lt/HansensDataset/DDA3//results.csv</csv>
 +
 +
 +
===Mauch's dataset===
 +
 +
====Summary Results====
 +
<csv>2020/lt/MauchsDataset/summary_MauchsDataset.csv</csv>
 +
 +
====Per-track results====
 +
GGL1
 +
<csv>2020/lt/MauchsDataset/GGL1//results.csv</csv>
 +
 +
GGL2
 +
<csv>2020/lt/MauchsDataset/GGL2//results.csv</csv>
 +
 +
RB1
 +
<csv>2020/lt/MauchsDataset/RB1//results.csv</csv>
 +
 +
DDA2
 +
<csv>2020/lt/MauchsDataset/DDA2//results.csv</csv>
 +
 +
DDA3
 +
<csv>2020/lt/MauchsDataset/DDA3//results.csv</csv>
 +
 +
 +
===Jamendo dataset===
 +
 +
====Summary Results====
 +
<csv>2020/lt/jamendolyrics/summary_jamendolyrics.csv</csv>
 +
 +
====Per-track results====
 +
 +
GGL1
 +
<csv>2020/lt/jamendolyrics/GGL1//results.csv</csv>
 +
 +
GGL2
 +
<csv>2020/lt/jamendolyrics/GGL2//results.csv</csv>
 +
 +
RB1
 +
<csv>2020/lt/jamendolyrics/RB1//results.csv</csv>
 +
 +
DDA2
 +
<csv>2020/lt/jamendolyrics/DDA2//results.csv</csv>
 +
 +
DDA3
 +
<csv>2020/lt/jamendolyrics/DDA3//results.csv</csv>

Revision as of 13:49, 13 February 2022

General Legend

Sub code Submission name Abstract Contributors
YYHL1 NetEase PDF Zhen Yang, Qichen Han, Xiang Li, Dong Liu, Peng Li
YYHL3 NetEase PDF Zhen Yang, Qichen Han, Xiang Li, Dong Liu, Peng Li


Results

Hansen's dataset a cappella

Summary Results

| border="1" cellspacing="0" style="text-align: left; width: 800px;" 

   |- style="background: white"
   ! width="200" | Submission
   ! width="100" | Avrg WER
   |-
   ! RB1| Sheffield University
   |-

Per-track results

GGL1 file /nema-raid/www/mirex/results/2020/lt/HansensDataset/YYHL1/results.csv not found

GGL2 file /nema-raid/www/mirex/results/2020/lt/HansensDataset/YYHL3/results.csv not found

RB1

Track WER Insertions Deletions Substitutions TotalWords
viva_la_vida 92.55 1 129 131 282
sunrise 75.20 13 11 70 125
this_afternoon 94.25 1 310 115 452
trick_me 86.55 1 125 125 290
rolling_in_the_deep 60.83 16 91 112 360
rehab 89.54 5 42 227 306
umbrella 97.57 0 339 62 411
Madonna-Beautiful_Stranger_full_ending 75.67 23 33 171 300
clocks_cut 78.82 3 12 52 85
i_kissed_a_girl 83.33 13 91 116 264

download these results as csv

DDA2

Track WER Insertions Deletions Substitutions TotalWords
clocks_cut 61.176 12 3 37 85
i_kissed_a_girl 79.167 34 10 165 264
Madonna-Beautiful_Stranger_full_ending 70.000 42 22 146 300
rehab 84.967 48 6 206 306
rolling_in_the_deep 44.444 24 12 124 360
sunrise 83.200 21 1 82 125
this_afternoon 95.796 96 7 330 452
trick_me 80.690 18 44 172 290
umbrella 69.586 35 55 196 411
viva_la_vida 79.078 25 10 188 282

download these results as csv

DDA3

Track WER Insertions Deletions Substitutions TotalWords
clocks_cut 75.294 2 8 54 85
i_kissed_a_girl 69.318 10 67 106 264
Madonna-Beautiful_Stranger_full_ending 82.333 1 131 115 300
rehab 80.392 1 136 109 306
rolling_in_the_deep 54.722 7 69 121 360
sunrise 84.000 26 8 71 125
this_afternoon 85.177 9 193 183 452
trick_me 82.414 0 95 144 290
umbrella 75.912 3 160 149 411
viva_la_vida 84.043 3 140 94 282

download these results as csv


Hansen's dataset

Summary Results

Submission Avg WER Avg insertions Avg Deletions Avg Substitutions Avg TotalWords
GGL1 47.86 6.40 65.10 67.80 287.50
GGL2 47.86 4.10 90.50 43.20 287.50
RB1 83.43 7.60 118.30 118.10 287.50
DDA2 74.81 35.50 17.00 164.60 287.50
DDA3 77.36 6.20 100.70 114.60 287.50

download these results as csv

Per-track results

GGL1

Track WER Insertions Deletions Substitutions TotalWords
clocks_cut 41.180 2 9 24 85
i_kissed_a_girl 29.170 9 10 58 264
Madonna-Beautiful_Stranger_full_ending 30.330 5 50 36 300
rehab 48.690 4 51 94 306
rolling_in_the_deep 47.220 8 61 101 360
sunrise 52.800 1 26 39 125
this_afternoon 54.870 25 67 156 452
trick_me 25.520 4 45 25 290
umbrella 49.150 6 62 134 411
viva_la_vida 99.650 0 270 11 282

download these results as csv

GGL2

Track WER Insertions Deletions Substitutions TotalWords
clocks_cut 47.060 0 29 11 85
i_kissed_a_girl 59.090 3 135 18 264
Madonna-Beautiful_Stranger_full_ending 32.670 4 65 29 300
rehab 48.690 1 121 27 306
rolling_in_the_deep 48.060 5 111 57 360
sunrise 52.000 1 44 20 125
this_afternoon 51.110 17 81 133 452
trick_me 18.620 1 33 20 290
umbrella 54.260 8 138 77 411
viva_la_vida 67.020 1 148 40 282

download these results as csv

RB1

Track WER Insertions Deletions Substitutions TotalWords
viva_la_vida 92.55 1 129 131 282
sunrise 75.20 13 11 70 125
this_afternoon 94.25 1 310 115 452
trick_me 86.55 1 125 125 290
rolling_in_the_deep 60.83 16 91 112 360
rehab 89.54 5 42 227 306
umbrella 97.57 0 339 62 411
Madonna-Beautiful_Stranger_full_ending 75.67 23 33 171 300
clocks_cut 78.82 3 12 52 85
i_kissed_a_girl 83.33 13 91 116 264

download these results as csv

DDA2

Track WER Insertions Deletions Substitutions TotalWords
clocks_cut 61.176 12 3 37 85
i_kissed_a_girl 79.167 34 10 165 264
Madonna-Beautiful_Stranger_full_ending 70.000 42 22 146 300
rehab 84.967 48 6 206 306
rolling_in_the_deep 44.444 24 12 124 360
sunrise 83.200 21 1 82 125
this_afternoon 95.796 96 7 330 452
trick_me 80.690 18 44 172 290
umbrella 69.586 35 55 196 411
viva_la_vida 79.078 25 10 188 282

download these results as csv

DDA3

Track WER Insertions Deletions Substitutions TotalWords
clocks_cut 75.294 2 8 54 85
i_kissed_a_girl 69.318 10 67 106 264
Madonna-Beautiful_Stranger_full_ending 82.333 1 131 115 300
rehab 80.392 1 136 109 306
rolling_in_the_deep 54.722 7 69 121 360
sunrise 84.000 26 8 71 125
this_afternoon 85.177 9 193 183 452
trick_me 82.414 0 95 144 290
umbrella 75.912 3 160 149 411
viva_la_vida 84.043 3 140 94 282

download these results as csv


Mauch's dataset

Summary Results

Submission Avg WER Avg insertions Avg Deletions Avg Substitutions Avg TotalWords
GGL1 47.25 4.65 52.90 62.10 252.60
GGL2 49.54 4.20 61.60 58.65 252.60
RB1 84.98 7.25 84.35 121.45 252.60
DDA2 75.39 30.65 16.05 138.90 252.60
DDA3 80.66 5.35 102.70 94.60 252.60

download these results as csv

Per-track results

GGL1

Track WER Insertions Deletions Substitutions TotalWords
Abba_KnowingMeKnowingYou 32.110 20 11 30 190
Bangles_EternalFlame 34.620 1 54 35 260
Blondie_CallMe 75.490 2 77 75 204
Duffy_WarwickAvenue 22.350 4 6 49 264
DuranDuran_OrdinaryWorld 32.570 8 40 37 261
FranzFerdinand_DoYouWantTo 54.950 2 105 115 404
Martika_ToySoldiers 74.800 2 127 61 254
Muse_GuidingLight 48.650 2 19 33 111
OtisRedding_TheDockOfTheBay 42.540 5 24 48 181
p082m 42.150 1 71 65 325
p084m 54.270 8 68 102 328
Queen_WeAreTheChampions 30.810 9 15 33 185
RobertPalmer_AddictedToLove 65.060 6 106 117 352
Santana_BlackMagicWoman 54.170 3 22 40 120
SimonAndGarfunkel_Cecilia 77.130 7 73 92 223
TakeThat_BackForGood 37.570 7 50 85 378
TinaTurner_WhatsLoveGotToDoWithIt 54.180 4 77 94 323
Toto_Africa 18.490 1 9 39 265
U2_WithOrWithoutYou 47.120 0 51 39 191
Zweieck_She 45.920 1 53 53 233

download these results as csv

GGL2

Track WER Insertions Deletions Substitutions TotalWords
Abba_KnowingMeKnowingYou 28.420 23 7 24 190
Bangles_EternalFlame 37.690 7 61 30 260
Blondie_CallMe 81.370 0 106 60 204
Duffy_WarwickAvenue 23.110 1 9 51 264
DuranDuran_OrdinaryWorld 38.700 8 44 49 261
FranzFerdinand_DoYouWantTo 55.450 3 102 119 404
Martika_ToySoldiers 77.560 2 131 64 254
Muse_GuidingLight 55.860 2 21 39 111
OtisRedding_TheDockOfTheBay 41.440 7 27 41 181
p082m 46.770 1 73 78 325
p084m 54.270 4 81 93 328
Queen_WeAreTheChampions 35.680 5 16 45 185
RobertPalmer_AddictedToLove 63.920 4 129 92 352
Santana_BlackMagicWoman 60.830 1 38 34 120
SimonAndGarfunkel_Cecilia 72.650 5 90 67 223
TakeThat_BackForGood 38.890 5 63 79 378
TinaTurner_WhatsLoveGotToDoWithIt 57.890 4 89 94 323
Toto_Africa 18.110 1 9 38 265
U2_WithOrWithoutYou 59.690 0 88 26 191
Zweieck_She 42.490 1 48 50 233

download these results as csv

RB1

Track WER Insertions Deletions Substitutions TotalWords
FranzFerdinand.DoYouWantTo 92.08 1 255 116 404
Zweieck.She 78.11 5 44 133 233
Toto.Africa 92.45 0 136 109 265
TinaTurner.WhatsLoveGotToDoWithIt 81.11 8 68 186 323
DuranDuran.OrdinaryWorld 91.95 0 130 110 261
Blondie.CallMe 87.75 2 51 126 204
p082m 72.31 25 24 186 325
Santana.BlackMagicWoman 92.50 1 18 92 120
Muse.GuidingLight 101.80 26 0 87 111
p084m 82.62 5 78 188 328
SimonAndGarfunkel.Cecilia 91.93 1 133 71 223
OtisRedding.TheDockOfTheBay 84.53 18 15 120 181
TakeThat.BackForGood 84.66 3 134 183 378
U2.WithOrWithoutYou 85.34 2 39 122 191
Martika.ToySoldiers 89.37 8 97 122 254
Bangles.EternalFlame 71.92 12 63 112 260
Duffy.WarwickAvenue 57.20 19 18 114 264
RobertPalmer.AddictedToLove 93.18 0 243 85 352
Abba.KnowingMeKnowingYou 93.16 1 98 78 190
Queen.WeAreTheChampions 75.68 8 43 89 185

download these results as csv

DDA2

Track WER Insertions Deletions Substitutions TotalWords
Abba.KnowingMeKnowingYou 66.842 44 1 82 190
Bangles.EternalFlame 66.538 16 29 128 260
Blondie.CallMe 100.490 39 1 165 204
Duffy.WarwickAvenue 52.273 32 3 103 264
DuranDuran.OrdinaryWorld 67.816 37 20 120 261
FranzFerdinand.DoYouWantTo 76.485 31 51 227 404
Martika.ToySoldiers 84.252 3 41 170 254
Muse.GuidingLight 105.405 44 0 73 111
OtisRedding.TheDockOfTheBay 74.033 22 6 106 181
p082m 69.538 59 15 152 325
p084m 71.951 28 13 195 328
Queen.WeAreTheChampions 65.405 15 2 104 185
RobertPalmer.AddictedToLove 80.398 61 17 205 352
Santana.BlackMagicWoman 103.333 66 0 58 120
SimonAndGarfunkel.Cecilia 89.238 8 30 161 223
TakeThat.BackForGood 65.608 22 20 206 378
TinaTurner.WhatsLoveGotToDoWithIt 61.920 19 42 139 323
Toto.Africa 78.491 42 8 158 265
U2.WithOrWithoutYou 59.162 16 8 89 191
Zweieck.She 68.670 9 14 137 233

download these results as csv

DDA3

Track WER Insertions Deletions Substitutions TotalWords
Abba.KnowingMeKnowingYou 52.105 6 28 65 190
Bangles.EternalFlame 82.308 1 116 97 260
Blondie.CallMe 95.098 0 160 34 204
Duffy.WarwickAvenue 63.258 8 55 104 264
DuranDuran.OrdinaryWorld 78.544 7 117 81 261
FranzFerdinand.DoYouWantTo 88.861 0 305 54 404
Martika.ToySoldiers 92.913 0 183 53 254
Muse.GuidingLight 90.991 1 63 37 111
OtisRedding.TheDockOfTheBay 77.901 1 28 112 181
p082m 72.000 21 58 155 325
p084m 74.695 4 63 178 328
Queen.WeAreTheChampions 88.108 3 62 98 185
RobertPalmer.AddictedToLove 90.909 0 241 79 352
Santana.BlackMagicWoman 89.167 8 0 99 120
SimonAndGarfunkel.Cecilia 87.892 7 102 87 223
TakeThat.BackForGood 74.074 1 135 144 378
TinaTurner.WhatsLoveGotToDoWithIt 73.684 20 89 129 323
Toto.Africa 84.906 14 88 123 265
U2.WithOrWithoutYou 81.152 3 90 62 191
Zweieck.She 74.678 2 71 101 233

download these results as csv


Jamendo dataset

Summary Results

Submission Avg WER Avg insertions Avg Deletions Avg Substitutions Avg TotalWords
GGL1 60.98 11.75 67.15 93.10 283.85
GGL2 62.46 8.05 90.05 83.80 283.85
RB1 86.70 8.70 90.60 140.60 283.85
DDA2 72.14 34.45 20.65 134.30 283.85
DDA3 73.09 7.15 85.40 103.40 283.85

download these results as csv

Per-track results

GGL1

Track WER Insertions Deletions Substitutions TotalWords
Avercage_-_Embers 89.950 0 119 51 189
Color_Out_-_Falling_Star 59.640 6 50 77 223
Cortez_-_Feel__Stripped_ 81.870 3 250 36 353
Explosive_Ear_Candy_-_Like_The_Sun 49.430 61 19 51 265
HILA_-_Give_Me_the_Same 46.580 6 66 78 322
JASON_MILLER_-_CROWD_PLEASER 49.230 21 43 192 520
Kinematic_-_Peyote 55.780 6 20 56 147
Lower_Loveday_-_Is_It_Right_ 47.640 10 15 76 212
LUNABLIND_-_Vision__Radio_Edit_ 28.120 12 18 51 288
Moon_I_Mean_-_Wrong_Concept 82.020 8 115 96 267
Pure_Mids_-_The_Leader 66.670 1 44 31 114
Quentin_Hannappe_-_Keep_On 54.860 3 39 54 175
Ridgway_-_Fire_Inside 58.220 4 71 102 304
Rxbyn_-_Bad_Side 67.500 8 169 120 440
Slingshot_Miracle_-_Whistler 74.070 8 31 81 162
Songwriterz_-_Back_In_Time 47.660 9 30 73 235
The.madpix.project_-_One_Way_Street 63.930 33 11 73 183
The_Rinn_-_Voices__2017_Version_ 68.320 7 34 97 202
Tom_Orlando_-_The_One__feat._Tina_G_ 57.780 10 103 173 495
Wordsmith_-_The_Statement 70.400 19 96 294 581

download these results as csv

GGL2

Track WER Insertions Deletions Substitutions TotalWords
Avercage_-_Embers 84.660 0 113 47 189
Color_Out_-_Falling_Star 64.130 5 68 70 223
Cortez_-_Feel__Stripped_ 81.870 2 257 30 353
Explosive_Ear_Candy_-_Like_The_Sun 46.420 38 17 68 265
HILA_-_Give_Me_the_Same 46.890 6 80 65 322
JASON_MILLER_-_CROWD_PLEASER 60.380 14 72 228 520
Kinematic_-_Peyote 60.540 3 18 68 147
Lower_Loveday_-_Is_It_Right_ 49.530 5 28 72 212
LUNABLIND_-_Vision__Radio_Edit_ 30.900 14 17 58 288
Moon_I_Mean_-_Wrong_Concept 82.020 6 118 95 267
Pure_Mids_-_The_Leader 60.530 2 46 21 114
Quentin_Hannappe_-_Keep_On 60.570 2 48 56 175
Ridgway_-_Fire_Inside 58.880 6 82 91 304
Rxbyn_-_Bad_Side 75.910 2 236 96 440
Slingshot_Miracle_-_Whistler 73.460 5 41 73 162
Songwriterz_-_Back_In_Time 55.320 26 20 84 235
The.madpix.project_-_One_Way_Street 45.360 2 32 49 183
The_Rinn_-_Voices__2017_Version_ 62.380 8 40 78 202
Tom_Orlando_-_The_One__feat._Tina_G_ 67.470 6 198 130 495
Wordsmith_-_The_Statement 81.930 9 270 197 581

download these results as csv

RB1

Track WER Insertions Deletions Substitutions TotalWords
Songwriterz_-_Back_In_Time 83.40 8 36 152 235
Lower_Loveday_-_Is_It_Right_ 85.85 2 116 64 212
JASON_MILLER_-_CROWD_PLEASER 67.31 27 39 284 520
Quentin_Hannappe_-_Keep_On 80.00 2 52 86 175
Pure_Mids_-_The_Leader 100.00 0 109 5 114
Avercage_-_Embers 94.18 2 42 134 189
HILA_-_Give_Me_the_Same 74.53 5 109 126 322
Ridgway_-_Fire_Inside 91.45 0 184 94 304
Rxbyn_-_Bad_Side 91.59 2 243 158 440
Tom_Orlando_-_The_One__feat._Tina_G_ 75.35 14 58 301 495
Color_Out_-_Falling_Star 90.58 2 111 89 223
Kinematic_-_Peyote 95.24 0 91 49 147
LUNABLIND_-_Vision__Radio_Edit_ 86.11 0 131 117 288
Cortez_-_Feel__Stripped_ 70.25 2 73 173 353
Moon_I_Mean_-_Wrong_Concept 91.76 2 66 177 267
Explosive_Ear_Candy_-_Like_The_Sun 83.02 35 32 153 265
The.madpix.project_-_One_Way_Street 84.70 33 16 106 183
The_Rinn_-_Voices__2017_Version_ 90.59 7 54 122 202
Wordsmith_-_The_Statement 91.22 7 249 274 581
Slingshot_Miracle_-_Whistler 106.79 24 1 148 162

download these results as csv

DDA2

Track WER Insertions Deletions Substitutions TotalWords
Avercage_-_Embers 90.476 0 37 134 189
Color_Out_-_Falling_Star 81.166 44 1 136 223
Cortez_-_Feel__Stripped_ 45.042 17 24 118 353
Explosive_Ear_Candy_-_Like_The_Sun 55.094 77 8 61 265
HILA_-_Give_Me_the_Same 58.385 37 25 126 322
JASON_MILLER_-_CROWD_PLEASER 46.538 38 26 178 520
Kinematic_-_Peyote 89.116 41 5 85 147
Lower_Loveday_-_Is_It_Right_ 75.472 54 2 104 212
LUNABLIND_-_Vision__Radio_Edit_ 53.472 42 11 101 288
Moon_I_Mean_-_Wrong_Concept 95.131 49 6 199 267
Pure_Mids_-_The_Leader 85.088 11 10 76 114
Quentin_Hannappe_-_Keep_On 54.857 15 8 73 175
Ridgway_-_Fire_Inside 62.171 18 35 136 304
Rxbyn_-_Bad_Side 78.409 13 109 223 440
Slingshot_Miracle_-_Whistler 124.074 73 0 128 162
Songwriterz_-_Back_In_Time 67.234 25 4 129 235
The.madpix.project_-_One_Way_Street 87.978 60 10 91 183
The_Rinn_-_Voices__2017_Version_ 82.673 27 5 135 202
Tom_Orlando_-_The_One__feat._Tina_G_ 61.818 21 60 225 495
Wordsmith_-_The_Statement 48.537 27 27 228 581

download these results as csv

DDA3

Track WER Insertions Deletions Substitutions TotalWords
Avercage_-_Embers 97.354 0 141 43 189
Color_Out_-_Falling_Star 86.099 3 118 71 223
Cortez_-_Feel__Stripped_ 54.391 8 78 106 353
Explosive_Ear_Candy_-_Like_The_Sun 58.113 13 54 87 265
HILA_-_Give_Me_the_Same 67.081 4 113 99 322
JASON_MILLER_-_CROWD_PLEASER 48.846 27 27 200 520
Kinematic_-_Peyote 91.837 2 71 62 147
Lower_Loveday_-_Is_It_Right_ 76.887 4 117 42 212
LUNABLIND_-_Vision__Radio_Edit_ 66.667 5 89 98 288
Moon_I_Mean_-_Wrong_Concept 83.895 3 83 138 267
Pure_Mids_-_The_Leader 85.965 0 66 32 114
Quentin_Hannappe_-_Keep_On 63.429 15 10 86 175
Ridgway_-_Fire_Inside 70.066 10 86 117 304
Rxbyn_-_Bad_Side 68.409 2 185 114 440
Slingshot_Miracle_-_Whistler 87.654 0 86 56 162
Songwriterz_-_Back_In_Time 54.894 2 44 83 235
The.madpix.project_-_One_Way_Street 73.770 6 39 90 183
The_Rinn_-_Voices__2017_Version_ 97.030 0 153 43 202
Tom_Orlando_-_The_One__feat._Tina_G_ 74.545 8 103 258 495
Wordsmith_-_The_Statement 54.905 31 45 243 581

download these results as csv

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2021:_Automatic_Lyrics_Transcription_Results&oldid=13556"