Difference between revisions of "2019:Automatic Lyrics-to-Audio Alignment Results"

From MIREX Wiki
(→‎Mauch's dataset)
(→‎Gracenote dataset)
Line 85: Line 85:
  
 
====Summary Results====
 
====Summary Results====
−
 
+
<csv>2019/ala/Gracenote/summary_Gracenote.csv</csv>
−
<csv>2019/ala/gracenote/summary_gracenote.csv</csv>
 
  
 
====Per-track results====
 
====Per-track results====
  
−
CW2
+
GYL1
−
<csv>2019/ala/gracenote/CW2_gracenote.csv</csv>
+
<csv>2019/ala/Gracenote/GYL1/metrics/Gracenote_GYL1.csv</csv>
−
 
 
−
CW3
 
−
<csv>2019/ala/gracenote/CW3_gracenote.csv</csv>
 
−
 
 
−
GSLW1
 
−
<csv>2019/ala/gracenote/GSLW1_gracenote.csv</csv>
 
−
 
 
−
GSLW2
 
−
<csv>2019/ala/gracenote/GSLW2_gracenote.csv</csv>
 
  
−
GSLW3
+
ED1
−
<csv>2019/ala/gracenote/GSLW3_gracenote.csv</csv>
+
<csv>2019/ala/Gracenote/ED1/metrics/Gracenote_ED1.csv</csv>

Revision as of 07:58, 5 November 2019

General Legend

Sub code Submission name Abstract Contributors
GYL1 NUS PDF Chitralekha Gupta, Haizhou Li, Emre Yilmaz
ED1 Audio2Lyrics_emirdemirel PDF Emir Demirel
SDE1 PDF Daniel Stoller, Simon Durand, Sebastian Ewert
SDE2 PDF Daniel Stoller, Simon Durand, Sebastian Ewert
LJ2 Force Alignment with Kaldi PDF Chuan You Lin, Jyh-Shing Roger Jang, Huang Hsiang-Yu

Results

Hansen's dataset a cappella

Summary Results

Submission Mean AAE Mean MAE Mean PCS Mean PCETW
GYL1 0.13 0.03 0.79 0.97
ED1 0.28 0.04 0.74 0.93
SDE2 0.21 0.08 0.47 0.94

download these results as csv

Per-track results

GYL1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
clocks_cut 0.287 0.023 0.750 0.953
i_kissed_a_girl 0.096 0.053 0.786 0.947
Madonna-Beautiful_Stranger_full_ending 0.076 0.034 0.787 0.960
rehab 0.035 0.018 0.905 0.990
rolling_in_the_deep 0.133 0.038 0.646 0.939
sunrise 0.515 0.019 0.587 0.896
this_afternoon 0.034 0.028 0.903 1.000
trick_me 0.031 0.022 0.903 1.000
umbrella 0.055 0.041 0.738 0.988
viva_la_vida 0.054 0.033 0.911 0.979

download these results as csv

ED1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
clocks_cut 1.190 0.037 0.497 0.871
i_kissed_a_girl 0.102 0.053 0.769 0.966
Madonna-Beautiful_Stranger_full_ending 0.110 0.042 0.757 0.923
rehab 0.036 0.020 0.935 0.993
rolling_in_the_deep 0.371 0.046 0.624 0.858
sunrise 0.644 0.034 0.591 0.832
this_afternoon 0.068 0.035 0.822 0.976
trick_me 0.152 0.033 0.703 0.938
umbrella 0.111 0.053 0.740 0.944
viva_la_vida 0.054 0.030 0.915 0.979

download these results as csv

SDE2

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
clocks_cut 0.288 0.073 0.507 0.953
i_kissed_a_girl 0.087 0.077 0.556 1.000
Madonna-Beautiful_Stranger_full_ending 0.101 0.067 0.405 0.963
rehab 0.130 0.083 0.490 0.958
rolling_in_the_deep 0.173 0.067 0.461 0.939
sunrise 0.542 0.069 0.449 0.840
this_afternoon 0.098 0.077 0.513 0.987
trick_me 0.276 0.096 0.407 0.914
umbrella 0.225 0.104 0.351 0.832
viva_la_vida 0.137 0.071 0.610 0.982

download these results as csv


Hansen's dataset

Summary Results

Submission Mean AAE Mean MAE Mean PCS Mean PCETW
GYL1 0.10 0.04 0.78 0.97
ED1 3.47 0.26 0.49 0.69
SDE2 0.39 0.09 0.44 0.88

download these results as csv

Per-track results

GYL1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
clocks_cut 0.028 0.021 0.801 1.000
i_kissed_a_girl 0.060 0.051 0.858 0.992
Madonna-Beautiful_Stranger_full_ending 0.080 0.036 0.765 0.933
rehab 0.049 0.042 0.850 1.000
rolling_in_the_deep 0.111 0.035 0.709 0.964
sunrise 0.348 0.046 0.702 0.888
this_afternoon 0.049 0.041 0.869 1.000
trick_me 0.158 0.051 0.648 0.959
umbrella 0.043 0.028 0.770 0.985
viva_la_vida 0.088 0.085 0.858 0.993

download these results as csv

ED1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
clocks_cut 2.046 0.042 0.567 0.847
i_kissed_a_girl 0.202 0.074 0.683 0.894
Madonna-Beautiful_Stranger_full_ending 11.740 0.244 0.379 0.530
rehab 0.195 0.053 0.736 0.895
rolling_in_the_deep 1.238 0.067 0.462 0.689
sunrise 0.745 0.065 0.539 0.776
this_afternoon 7.727 0.581 0.331 0.469
trick_me 2.304 0.103 0.373 0.655
umbrella 0.841 0.059 0.565 0.742
viva_la_vida 7.645 1.307 0.227 0.394

download these results as csv

SDE2

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
clocks_cut 0.205 0.071 0.560 0.929
i_kissed_a_girl 0.107 0.085 0.550 0.977
Madonna-Beautiful_Stranger_full_ending 0.164 0.078 0.375 0.910
rehab 0.242 0.095 0.417 0.882
rolling_in_the_deep 0.185 0.071 0.441 0.917
sunrise 0.938 0.082 0.431 0.784
this_afternoon 0.140 0.092 0.433 0.934
trick_me 0.600 0.122 0.248 0.728
umbrella 1.212 0.113 0.301 0.742
viva_la_vida 0.115 0.080 0.624 0.961

download these results as csv

Mauch's dataset

Summary Results

Submission Mean AAE Mean MAE Mean PCS Mean PCETW
GYL1 0.19 0.10 NA 0.91
ED1 11.65 8.46 NA 0.39
SDE1 0.34 0.11 NA 0.88
SDE2 0.26 0.11 NA 0.87

download these results as csv

Per-track results

GYL1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance windowAbba.KnowingMeKnowingYou 0.073 0.060 NaN 0.989
Bangles.EternalFlame 0.220 0.178 NaN 0.815
Blondie.CallMe 0.455 0.126 NaN 0.863
Duffy.WarwickAvenue 0.056 0.045 NaN 0.992
DuranDuran.OrdinaryWorld 0.108 0.070 NaN 0.969
FranzFerdinand.DoYouWantTo 0.362 0.129 NaN 0.743
Martika.ToySoldiers 0.149 0.105 NaN 0.898
Muse.GuidingLight 0.201 0.059 NaN 0.964
OtisRedding.TheDockOfTheBay 0.205 0.165 NaN 0.873
p082m 0.091 0.055 NaN 0.985
p084m 0.062 0.051 NaN 0.997
Queen.WeAreTheChampions 0.134 0.059 NaN 0.930
RobertPalmer.AddictedToLove 0.157 0.148 NaN 0.963
Santana.BlackMagicWoman 0.203 0.171 NaN 0.800
SimonAndGarfunkel.Cecilia 0.566 0.081 NaN 0.659
TakeThat.BackForGood 0.122 0.047 NaN 0.929
TinaTurner.WhatsLoveGotToDoWithIt 0.191 0.171 NaN 0.944
Toto.Africa 0.094 0.089 NaN 1.000
U2.WithOrWithoutYou 0.325 0.093 NaN 0.901
Zweieck.She 0.076 0.057 NaN 0.979

download these results as csv

ED1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
Abba.KnowingMeKnowingYou 17.805 17.976 NaN 0.053
Bangles.EternalFlame 29.498 26.443 NaN 0.138
Blondie.CallMe 5.397 1.464 NaN 0.348
Duffy.WarwickAvenue 30.261 20.208 NaN 0.182
DuranDuran.OrdinaryWorld 11.125 0.220 NaN 0.510
FranzFerdinand.DoYouWantTo 2.020 0.432 NaN 0.473
Martika.ToySoldiers 10.824 12.892 NaN 0.091
Muse.GuidingLight 8.475 6.233 NaN 0.252
OtisRedding.TheDockOfTheBay 2.242 0.260 NaN 0.624
p082m 25.593 30.146 NaN 0.166
p084m 1.715 0.074 NaN 0.777
Queen.WeAreTheChampions 4.691 0.825 NaN 0.422
RobertPalmer.AddictedToLove 1.285 0.165 NaN 0.651
Santana.BlackMagicWoman 22.247 17.685 NaN 0.058
SimonAndGarfunkel.Cecilia 2.680 0.495 NaN 0.466
TakeThat.BackForGood 1.390 0.108 NaN 0.698
TinaTurner.WhatsLoveGotToDoWithIt 0.910 0.144 NaN 0.672
Toto.Africa 9.599 4.183 NaN 0.362
U2.WithOrWithoutYou 44.660 29.011 NaN 0.267
Zweieck.She 0.669 0.144 NaN 0.644

download these results as csv

SDE1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
Abba.KnowingMeKnowingYou 0.124 0.069 NaN 0.963
Bangles.EternalFlame 0.141 0.120 NaN 0.946
Blondie.CallMe 0.258 0.084 NaN 0.877
Duffy.WarwickAvenue 0.094 0.074 NaN 0.989
DuranDuran.OrdinaryWorld 0.184 0.068 NaN 0.950
FranzFerdinand.DoYouWantTo 0.841 0.128 NaN 0.710
Martika.ToySoldiers 0.512 0.102 NaN 0.768
Muse.GuidingLight 0.481 0.126 NaN 0.901
OtisRedding.TheDockOfTheBay 0.217 0.198 NaN 0.801
p082m 0.130 0.071 NaN 0.957
p084m 0.139 0.071 NaN 0.915
Queen.WeAreTheChampions 0.174 0.086 NaN 0.908
RobertPalmer.AddictedToLove 0.174 0.105 NaN 0.935
Santana.BlackMagicWoman 0.683 0.127 NaN 0.783
SimonAndGarfunkel.Cecilia 0.731 0.152 NaN 0.583
TakeThat.BackForGood 0.128 0.068 NaN 0.926
TinaTurner.WhatsLoveGotToDoWithIt 0.181 0.139 NaN 0.882
Toto.Africa 0.216 0.139 NaN 0.940
U2.WithOrWithoutYou 1.182 0.100 NaN 0.890
Zweieck.She 0.153 0.073 NaN 0.936

download these results as csv

SDE2

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
Abba.KnowingMeKnowingYou 0.182 0.091 NaN 0.916
Bangles.EternalFlame 0.207 0.169 NaN 0.862
Blondie.CallMe 0.504 0.115 NaN 0.843
Duffy.WarwickAvenue 0.086 0.066 NaN 0.989
DuranDuran.OrdinaryWorld 0.103 0.074 NaN 0.950
FranzFerdinand.DoYouWantTo 0.263 0.132 NaN 0.802
Martika.ToySoldiers 0.375 0.137 NaN 0.740
Muse.GuidingLight 0.444 0.091 NaN 0.937
OtisRedding.TheDockOfTheBay 0.176 0.159 NaN 0.878
p082m 0.166 0.064 NaN 0.972
p084m 0.130 0.066 NaN 0.924
Queen.WeAreTheChampions 0.199 0.080 NaN 0.892
RobertPalmer.AddictedToLove 0.226 0.149 NaN 0.864
Santana.BlackMagicWoman 0.231 0.174 NaN 0.717
SimonAndGarfunkel.Cecilia 0.822 0.153 NaN 0.641
TakeThat.BackForGood 0.127 0.082 NaN 0.929
TinaTurner.WhatsLoveGotToDoWithIt 0.211 0.178 NaN 0.839
Toto.Africa 0.133 0.105 NaN 0.947
U2.WithOrWithoutYou 0.413 0.092 NaN 0.916
Zweieck.She 0.124 0.073 NaN 0.931

download these results as csv

Gracenote dataset

Summary Results

Submission Mean AAE Mean MAE Mean PCS Mean PCETW
GYL1 0.09 0.04 NA 0.96
ED1 0.23 0.06 NA 0.85

download these results as csv

Per-track results

GYL1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance windowABBA-Dancing_Queen 0.132 0.059 NaN 0.945
Alanis_Morrisette-One_hand_in_my_pocket 0.049 0.030 NaN 1.000
Daft_Punk-Harder_Faster 0.139 0.030 NaN 0.927
Metalica-Enter_Sandman 0.071 0.057 NaN 0.980
Michael_Jackson-The_way_you_make_me_feel 0.062 0.043 NaN 0.991
Notorious_Big-Big_poppa 0.052 0.035 NaN 0.979
Smashing_Pumpkins-1979 0.192 0.053 NaN 0.891
Taylor_Swift-Shake_it_off 0.059 0.037 NaN 0.964

download these results as csv

ED1

Track Average absolute error Median absolute error Percentage of correct segments Percentage of correct estimates with tolerance window
ABBA-Dancing_Queen 0.658 0.078 NaN 0.616
Alanis_Morrisette-One_hand_in_my_pocket 0.064 0.032 NaN 0.968
Daft_Punk-Harder_Faster 0.097 0.029 NaN 0.964
Metalica-Enter_Sandman 0.159 0.033 NaN 0.918
Michael_Jackson-The_way_you_make_me_feel 0.147 0.059 NaN 0.841
Notorious_Big-Big_poppa 0.072 0.043 NaN 0.962
Smashing_Pumpkins-1979 0.524 0.186 NaN 0.674
Taylor_Swift-Shake_it_off 0.145 0.045 NaN 0.838

download these results as csv

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2019:Automatic_Lyrics-to-Audio_Alignment_Results&oldid=13134"