Difference between revisions of "2019:Automatic Lyrics-to-Audio Alignment Results"

From MIREX Wiki
(→‎General Legend)
(→‎General Legend)
 
Line 18: Line 18:
  
 
     ! SDE1
 
     ! SDE1
−
     | ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2019/SDE1.pdf PDF] || [http://www.eecs.qmul.ac.uk/profiles/stollerdaniel.html Daniel Stoller], [https://scholar.google.com/citations?user=N2oBCKkAAAAJ&hl=en Simon Durand], [https://sebewert.github.io/ Sebastian Ewert]
+
     | SDE1 ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2019/SDE1.pdf PDF] || [http://www.eecs.qmul.ac.uk/profiles/stollerdaniel.html Daniel Stoller], [https://scholar.google.com/citations?user=N2oBCKkAAAAJ&hl=en Simon Durand], [https://sebewert.github.io/ Sebastian Ewert]
 
     |-
 
     |-
  
 
     ! SDE2
 
     ! SDE2
−
     | ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2019/SDE2.pdf PDF] || [http://www.eecs.qmul.ac.uk/profiles/stollerdaniel.html Daniel Stoller], [https://scholar.google.com/citations?user=N2oBCKkAAAAJ&hl=en Simon Durand], [https://sebewert.github.io/ Sebastian Ewert]
+
     | SDE2 ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2019/SDE2.pdf PDF] || [http://www.eecs.qmul.ac.uk/profiles/stollerdaniel.html Daniel Stoller], [https://scholar.google.com/citations?user=N2oBCKkAAAAJ&hl=en Simon Durand], [https://sebewert.github.io/ Sebastian Ewert]
 
     |-
 
     |-
  

Latest revision as of 18:45, 6 November 2019

General Legend

Sub code Submission name Abstract Contributors
GYL1 NUS PDF Chitralekha Gupta, Haizhou Li, Emre Yilmaz
ED1 Audio2Lyrics_emirdemirel PDF Emir Demirel
SDE1 SDE1 PDF Daniel Stoller, Simon Durand, Sebastian Ewert
SDE2 SDE2 PDF Daniel Stoller, Simon Durand, Sebastian Ewert

Results

Hansen's dataset a cappella

Summary Results

Submission Mean AAE Mean MAE Mean PCS Mean PCETW
GYL1 0.13 0.03 0.79 0.97
ED1 0.28 0.04 0.74 0.93
SDE2 0.21 0.08 0.47 0.94

download these results as csv

Per-track results

GYL1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
clocks_cut 0.287 0.023 0.750 0.953
i_kissed_a_girl 0.096 0.053 0.786 0.947
Madonna-Beautiful_Stranger_full_ending 0.076 0.034 0.787 0.960
rehab 0.035 0.018 0.905 0.990
rolling_in_the_deep 0.133 0.038 0.646 0.939
sunrise 0.515 0.019 0.587 0.896
this_afternoon 0.034 0.028 0.903 1.000
trick_me 0.031 0.022 0.903 1.000
umbrella 0.055 0.041 0.738 0.988
viva_la_vida 0.054 0.033 0.911 0.979

download these results as csv

ED1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
clocks_cut 1.190 0.037 0.497 0.871
i_kissed_a_girl 0.102 0.053 0.769 0.966
Madonna-Beautiful_Stranger_full_ending 0.110 0.042 0.757 0.923
rehab 0.036 0.020 0.935 0.993
rolling_in_the_deep 0.371 0.046 0.624 0.858
sunrise 0.644 0.034 0.591 0.832
this_afternoon 0.068 0.035 0.822 0.976
trick_me 0.152 0.033 0.703 0.938
umbrella 0.111 0.053 0.740 0.944
viva_la_vida 0.054 0.030 0.915 0.979

download these results as csv

SDE2

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
clocks_cut 0.288 0.073 0.507 0.953
i_kissed_a_girl 0.087 0.077 0.556 1.000
Madonna-Beautiful_Stranger_full_ending 0.101 0.067 0.405 0.963
rehab 0.130 0.083 0.490 0.958
rolling_in_the_deep 0.173 0.067 0.461 0.939
sunrise 0.542 0.069 0.449 0.840
this_afternoon 0.098 0.077 0.513 0.987
trick_me 0.276 0.096 0.407 0.914
umbrella 0.225 0.104 0.351 0.832
viva_la_vida 0.137 0.071 0.610 0.982

download these results as csv


Hansen's dataset

Summary Results

Submission Mean AAE Mean MAE Mean PCS Mean PCETW
GYL1 0.10 0.04 0.78 0.97
ED1 3.47 0.26 0.49 0.69
SDE2 0.39 0.09 0.44 0.88

download these results as csv

Per-track results

GYL1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
clocks_cut 0.028 0.021 0.801 1.000
i_kissed_a_girl 0.060 0.051 0.858 0.992
Madonna-Beautiful_Stranger_full_ending 0.080 0.036 0.765 0.933
rehab 0.049 0.042 0.850 1.000
rolling_in_the_deep 0.111 0.035 0.709 0.964
sunrise 0.348 0.046 0.702 0.888
this_afternoon 0.049 0.041 0.869 1.000
trick_me 0.158 0.051 0.648 0.959
umbrella 0.043 0.028 0.770 0.985
viva_la_vida 0.088 0.085 0.858 0.993

download these results as csv

ED1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
clocks_cut 2.046 0.042 0.567 0.847
i_kissed_a_girl 0.202 0.074 0.683 0.894
Madonna-Beautiful_Stranger_full_ending 11.740 0.244 0.379 0.530
rehab 0.195 0.053 0.736 0.895
rolling_in_the_deep 1.238 0.067 0.462 0.689
sunrise 0.745 0.065 0.539 0.776
this_afternoon 7.727 0.581 0.331 0.469
trick_me 2.304 0.103 0.373 0.655
umbrella 0.841 0.059 0.565 0.742
viva_la_vida 7.645 1.307 0.227 0.394

download these results as csv

SDE2

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
clocks_cut 0.205 0.071 0.560 0.929
i_kissed_a_girl 0.107 0.085 0.550 0.977
Madonna-Beautiful_Stranger_full_ending 0.164 0.078 0.375 0.910
rehab 0.242 0.095 0.417 0.882
rolling_in_the_deep 0.185 0.071 0.441 0.917
sunrise 0.938 0.082 0.431 0.784
this_afternoon 0.140 0.092 0.433 0.934
trick_me 0.600 0.122 0.248 0.728
umbrella 1.212 0.113 0.301 0.742
viva_la_vida 0.115 0.080 0.624 0.961

download these results as csv

Mauch's dataset

Summary Results

Submission Mean AAE Mean MAE Mean PCS Mean PCETW
GYL1 0.19 0.10 NA 0.91
ED1 11.65 8.46 NA 0.39
SDE1 0.34 0.11 NA 0.88
SDE2 0.26 0.11 NA 0.87

download these results as csv

Per-track results

GYL1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance windowAbba.KnowingMeKnowingYou 0.073 0.060 NaN 0.989
Bangles.EternalFlame 0.220 0.178 NaN 0.815
Blondie.CallMe 0.455 0.126 NaN 0.863
Duffy.WarwickAvenue 0.056 0.045 NaN 0.992
DuranDuran.OrdinaryWorld 0.108 0.070 NaN 0.969
FranzFerdinand.DoYouWantTo 0.362 0.129 NaN 0.743
Martika.ToySoldiers 0.149 0.105 NaN 0.898
Muse.GuidingLight 0.201 0.059 NaN 0.964
OtisRedding.TheDockOfTheBay 0.205 0.165 NaN 0.873
p082m 0.091 0.055 NaN 0.985
p084m 0.062 0.051 NaN 0.997
Queen.WeAreTheChampions 0.134 0.059 NaN 0.930
RobertPalmer.AddictedToLove 0.157 0.148 NaN 0.963
Santana.BlackMagicWoman 0.203 0.171 NaN 0.800
SimonAndGarfunkel.Cecilia 0.566 0.081 NaN 0.659
TakeThat.BackForGood 0.122 0.047 NaN 0.929
TinaTurner.WhatsLoveGotToDoWithIt 0.191 0.171 NaN 0.944
Toto.Africa 0.094 0.089 NaN 1.000
U2.WithOrWithoutYou 0.325 0.093 NaN 0.901
Zweieck.She 0.076 0.057 NaN 0.979

download these results as csv

ED1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
Abba.KnowingMeKnowingYou 17.805 17.976 NaN 0.053
Bangles.EternalFlame 29.498 26.443 NaN 0.138
Blondie.CallMe 5.397 1.464 NaN 0.348
Duffy.WarwickAvenue 30.261 20.208 NaN 0.182
DuranDuran.OrdinaryWorld 11.125 0.220 NaN 0.510
FranzFerdinand.DoYouWantTo 2.020 0.432 NaN 0.473
Martika.ToySoldiers 10.824 12.892 NaN 0.091
Muse.GuidingLight 8.475 6.233 NaN 0.252
OtisRedding.TheDockOfTheBay 2.242 0.260 NaN 0.624
p082m 25.593 30.146 NaN 0.166
p084m 1.715 0.074 NaN 0.777
Queen.WeAreTheChampions 4.691 0.825 NaN 0.422
RobertPalmer.AddictedToLove 1.285 0.165 NaN 0.651
Santana.BlackMagicWoman 22.247 17.685 NaN 0.058
SimonAndGarfunkel.Cecilia 2.680 0.495 NaN 0.466
TakeThat.BackForGood 1.390 0.108 NaN 0.698
TinaTurner.WhatsLoveGotToDoWithIt 0.910 0.144 NaN 0.672
Toto.Africa 9.599 4.183 NaN 0.362
U2.WithOrWithoutYou 44.660 29.011 NaN 0.267
Zweieck.She 0.669 0.144 NaN 0.644

download these results as csv

SDE1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
Abba.KnowingMeKnowingYou 0.124 0.069 NaN 0.963
Bangles.EternalFlame 0.141 0.120 NaN 0.946
Blondie.CallMe 0.258 0.084 NaN 0.877
Duffy.WarwickAvenue 0.094 0.074 NaN 0.989
DuranDuran.OrdinaryWorld 0.184 0.068 NaN 0.950
FranzFerdinand.DoYouWantTo 0.841 0.128 NaN 0.710
Martika.ToySoldiers 0.512 0.102 NaN 0.768
Muse.GuidingLight 0.481 0.126 NaN 0.901
OtisRedding.TheDockOfTheBay 0.217 0.198 NaN 0.801
p082m 0.130 0.071 NaN 0.957
p084m 0.139 0.071 NaN 0.915
Queen.WeAreTheChampions 0.174 0.086 NaN 0.908
RobertPalmer.AddictedToLove 0.174 0.105 NaN 0.935
Santana.BlackMagicWoman 0.683 0.127 NaN 0.783
SimonAndGarfunkel.Cecilia 0.731 0.152 NaN 0.583
TakeThat.BackForGood 0.128 0.068 NaN 0.926
TinaTurner.WhatsLoveGotToDoWithIt 0.181 0.139 NaN 0.882
Toto.Africa 0.216 0.139 NaN 0.940
U2.WithOrWithoutYou 1.182 0.100 NaN 0.890
Zweieck.She 0.153 0.073 NaN 0.936

download these results as csv

SDE2

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
Abba.KnowingMeKnowingYou 0.182 0.091 NaN 0.916
Bangles.EternalFlame 0.207 0.169 NaN 0.862
Blondie.CallMe 0.504 0.115 NaN 0.843
Duffy.WarwickAvenue 0.086 0.066 NaN 0.989
DuranDuran.OrdinaryWorld 0.103 0.074 NaN 0.950
FranzFerdinand.DoYouWantTo 0.263 0.132 NaN 0.802
Martika.ToySoldiers 0.375 0.137 NaN 0.740
Muse.GuidingLight 0.444 0.091 NaN 0.937
OtisRedding.TheDockOfTheBay 0.176 0.159 NaN 0.878
p082m 0.166 0.064 NaN 0.972
p084m 0.130 0.066 NaN 0.924
Queen.WeAreTheChampions 0.199 0.080 NaN 0.892
RobertPalmer.AddictedToLove 0.226 0.149 NaN 0.864
Santana.BlackMagicWoman 0.231 0.174 NaN 0.717
SimonAndGarfunkel.Cecilia 0.822 0.153 NaN 0.641
TakeThat.BackForGood 0.127 0.082 NaN 0.929
TinaTurner.WhatsLoveGotToDoWithIt 0.211 0.178 NaN 0.839
Toto.Africa 0.133 0.105 NaN 0.947
U2.WithOrWithoutYou 0.413 0.092 NaN 0.916
Zweieck.She 0.124 0.073 NaN 0.931

download these results as csv

Gracenote dataset

Summary Results

Submission Mean AAE Mean MAE Mean PCS Mean PCETW
GYL1 0.09 0.04 NA 0.96
ED1 0.23 0.06 NA 0.85

download these results as csv

Per-track results

GYL1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance windowABBA-Dancing_Queen 0.132 0.059 NaN 0.945
Alanis_Morrisette-One_hand_in_my_pocket 0.049 0.030 NaN 1.000
Daft_Punk-Harder_Faster 0.139 0.030 NaN 0.927
Metalica-Enter_Sandman 0.071 0.057 NaN 0.980
Michael_Jackson-The_way_you_make_me_feel 0.062 0.043 NaN 0.991
Notorious_Big-Big_poppa 0.052 0.035 NaN 0.979
Smashing_Pumpkins-1979 0.192 0.053 NaN 0.891
Taylor_Swift-Shake_it_off 0.059 0.037 NaN 0.964

download these results as csv

ED1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
ABBA-Dancing_Queen 0.658 0.078 NaN 0.616
Alanis_Morrisette-One_hand_in_my_pocket 0.064 0.032 NaN 0.968
Daft_Punk-Harder_Faster 0.097 0.029 NaN 0.964
Metalica-Enter_Sandman 0.159 0.033 NaN 0.918
Michael_Jackson-The_way_you_make_me_feel 0.147 0.059 NaN 0.841
Notorious_Big-Big_poppa 0.072 0.043 NaN 0.962
Smashing_Pumpkins-1979 0.524 0.186 NaN 0.674
Taylor_Swift-Shake_it_off 0.145 0.045 NaN 0.838

download these results as csv


Jamendo dataset

Summary Results

Submission Mean AAE Mean MAE Mean PCS Mean PCETW
GYL1 0.22 0.05 NA 0.94
ED1 7.29 4.78 NA 0.57
SDE1 0.82 0.10 NA 0.85
SDE2 0.38 0.10 NA 0.87

download these results as csv

Per-track results

GYL1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
Avercage_-_Embers 0.753 0.088 NaN 0.831
Color_Out_-_Falling_Star 0.379 0.070 NaN 0.928
Cortez_-_Feel__Stripped_ 0.056 0.041 NaN 0.997
Explosive_Ear_Candy_-_Like_The_Sun 0.061 0.052 NaN 1.000
HILA_-_Give_Me_the_Same 0.142 0.046 NaN 0.963
JASON_MILLER_-_CROWD_PLEASER 0.050 0.038 NaN 0.996
Kinematic_-_Peyote 0.082 0.058 NaN 0.973
Lower_Loveday_-_Is_It_Right_ 0.062 0.042 NaN 0.986
LUNABLIND_-_Vision__Radio_Edit_ 0.043 0.031 NaN 1.000
Moon_I_Mean_-_Wrong_Concept 0.107 0.051 NaN 0.936
Pure_Mids_-_The_Leader 1.203 0.051 NaN 0.728
Quentin_Hannappe_-_Keep_On 0.085 0.037 NaN 0.971
Ridgway_-_Fire_Inside 0.449 0.041 NaN 0.852
Rxbyn_-_Bad_Side 0.088 0.041 NaN 0.961
Slingshot_Miracle_-_Whistler 0.211 0.047 NaN 0.969
Songwriterz_-_Back_In_Time 0.119 0.050 NaN 0.940
The.madpix.project_-_One_Way_Street 0.046 0.036 NaN 0.995
The_Rinn_-_Voices__2017_Version_ 0.170 0.039 NaN 0.941
Tom_Orlando_-_The_One__feat._Tina_G_ 0.201 0.039 NaN 0.929
Wordsmith_-_The_Statement 0.034 0.024 NaN 0.997

download these results as csv

ED1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
Avercage_-_Embers 30.859 30.024 NaN 0.000
Color_Out_-_Falling_Star 8.219 0.221 NaN 0.561
Cortez_-_Feel__Stripped_ 0.490 0.072 NaN 0.839
Explosive_Ear_Candy_-_Like_The_Sun 0.487 0.054 NaN 0.860
HILA_-_Give_Me_the_Same 0.370 0.075 NaN 0.773
JASON_MILLER_-_CROWD_PLEASER 0.079 0.040 NaN 0.963
Kinematic_-_Peyote 19.700 7.163 NaN 0.224
Lower_Loveday_-_Is_It_Right_ 6.463 3.263 NaN 0.368
LUNABLIND_-_Vision__Radio_Edit_ 2.334 0.084 NaN 0.639
Moon_I_Mean_-_Wrong_Concept 12.391 13.930 NaN 0.116
Pure_Mids_-_The_Leader 41.743 39.209 NaN 0.079
Quentin_Hannappe_-_Keep_On 1.573 0.097 NaN 0.594
Ridgway_-_Fire_Inside 5.106 0.079 NaN 0.704
Rxbyn_-_Bad_Side 2.566 0.168 NaN 0.539
Slingshot_Miracle_-_Whistler 1.126 0.147 NaN 0.580
Songwriterz_-_Back_In_Time 0.544 0.060 NaN 0.826
The.madpix.project_-_One_Way_Street 3.824 0.081 NaN 0.623
The_Rinn_-_Voices__2017_Version_ 5.656 0.773 NaN 0.411
Tom_Orlando_-_The_One__feat._Tina_G_ 2.176 0.072 NaN 0.701
Wordsmith_-_The_Statement 0.143 0.049 NaN 0.960

download these results as csv

SDE1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
Avercage_-_Embers 2.028 0.183 NaN 0.577
Color_Out_-_Falling_Star 0.533 0.145 NaN 0.897
Cortez_-_Feel__Stripped_ 0.166 0.101 NaN 0.935
Explosive_Ear_Candy_-_Like_The_Sun 0.123 0.038 NaN 0.951
HILA_-_Give_Me_the_Same 0.134 0.056 NaN 0.916
JASON_MILLER_-_CROWD_PLEASER 0.249 0.094 NaN 0.825
Kinematic_-_Peyote 0.075 0.051 NaN 0.973
Lower_Loveday_-_Is_It_Right_ 0.188 0.059 NaN 0.943
LUNABLIND_-_Vision__Radio_Edit_ 0.088 0.048 NaN 0.983
Moon_I_Mean_-_Wrong_Concept 0.330 0.065 NaN 0.888
Pure_Mids_-_The_Leader 7.034 0.106 NaN 0.763
Quentin_Hannappe_-_Keep_On 0.447 0.062 NaN 0.869
Ridgway_-_Fire_Inside 0.906 0.080 NaN 0.878
Rxbyn_-_Bad_Side 1.012 0.148 NaN 0.645
Slingshot_Miracle_-_Whistler 0.684 0.060 NaN 0.840
Songwriterz_-_Back_In_Time 0.108 0.058 NaN 0.915
The.madpix.project_-_One_Way_Street 0.291 0.057 NaN 0.962
The_Rinn_-_Voices__2017_Version_ 0.208 0.059 NaN 0.916
Tom_Orlando_-_The_One__feat._Tina_G_ 0.726 0.075 NaN 0.848
Wordsmith_-_The_Statement 1.038 0.397 NaN 0.427

download these results as csv

SDE2

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
Avercage_-_Embers 1.174 0.138 NaN 0.677
Color_Out_-_Falling_Star 0.194 0.114 NaN 0.910
Cortez_-_Feel__Stripped_ 0.107 0.081 NaN 0.977
Explosive_Ear_Candy_-_Like_The_Sun 0.072 0.065 NaN 1.000
HILA_-_Give_Me_the_Same 0.160 0.065 NaN 0.916
JASON_MILLER_-_CROWD_PLEASER 0.199 0.102 NaN 0.871
Kinematic_-_Peyote 0.074 0.057 NaN 0.980
Lower_Loveday_-_Is_It_Right_ 0.107 0.068 NaN 0.943
LUNABLIND_-_Vision__Radio_Edit_ 0.119 0.064 NaN 0.958
Moon_I_Mean_-_Wrong_Concept 0.118 0.067 NaN 0.929
Pure_Mids_-_The_Leader 2.014 0.083 NaN 0.746
Quentin_Hannappe_-_Keep_On 0.175 0.068 NaN 0.943
Ridgway_-_Fire_Inside 0.504 0.101 NaN 0.849
Rxbyn_-_Bad_Side 0.485 0.143 NaN 0.659
Slingshot_Miracle_-_Whistler 0.145 0.068 NaN 0.944
Songwriterz_-_Back_In_Time 0.145 0.066 NaN 0.911
The.madpix.project_-_One_Way_Street 0.394 0.066 NaN 0.934
The_Rinn_-_Voices__2017_Version_ 0.180 0.073 NaN 0.901
Tom_Orlando_-_The_One__feat._Tina_G_ 0.180 0.082 NaN 0.907
Wordsmith_-_The_Statement 0.958 0.369 NaN 0.448

download these results as csv

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2019:Automatic_Lyrics-to-Audio_Alignment_Results&oldid=13137"