Difference between revisions of "2018:Automatic Lyrics-to-Audio Alignment Results"

From MIREX Wiki
(→‎General Legend)
(→‎Subtask1: A cappella Mandarin Chinese pop songs)
 
(6 intermediate revisions by the same user not shown)
Line 10: Line 10:
  
 
     ! CW1
 
     ! CW1
−
     | HMM-based Alignment subtask1  ||  style="text-align: center;" | [PDF] || [mailto:chungchewang@kkbox.com Chung-Che Wang]
+
     | HMM-based Alignment subtask1  ||  style="text-align: center;" | [PDF not provided] || [mailto:chungchewang@kkbox.com Chung-Che Wang]
 
     |-
 
     |-
  
Line 33: Line 33:
 
     |-
 
     |-
  
−
     ! FFJ1(we were not able to run the code)
+
     ! FFJ1
−
     | subtask 1 & 2  ||  style="text-align: center;" |  [PDF] || [mailto:yangbih0914@gmail.com Yang-Bin Dell Fan], [mailto:lambert.fan@mirllab.org Zhe-Cheng Fan], [mailto:jang@csie.ntu.edu.tw Jyh-Shing Roger Jang]
+
     | subtask 1 & 2       [not able to run their code] ||  style="text-align: center;" |  [PDF not provided] || [mailto:yangbih0914@gmail.com Yang-Bin Dell Fan], [mailto:lambert.fan@mirllab.org Zhe-Cheng Fan], [mailto:jang@csie.ntu.edu.tw Jyh-Shing Roger Jang]
 
     |-
 
     |-
  
Line 42: Line 42:
 
==Subtask1: A cappella Mandarin Chinese pop songs==
 
==Subtask1: A cappella Mandarin Chinese pop songs==
 
===Source-separated===
 
===Source-separated===
−
===Mixed===
+
====Summary Results====
 +
<csv>2018/ala/kugou/clean/summary_kugou_clean.csv</csv>
 +
 
 +
====Per-track results====
 +
 
 +
CW1
 +
<csv>2018/ala/kugou/clean/CW1_kugou_clean.csv</csv>
 +
 
 +
===Mix===
 +
====Summary Results====
 +
<csv>2018/ala/kugou/mix/summary_kugou_mix.csv</csv>
 +
 
 +
====Per-track results====
 +
 
 +
CW1
 +
<csv>2018/ala/kugou/mix/CW1_kugou_mix.csv</csv>
  
 
==Subtask2: English pop songs==
 
==Subtask2: English pop songs==
Line 112: Line 127:
  
 
===Gracenote dataset===
 
===Gracenote dataset===
 +
 +
====Summary Results====
 +
 +
<csv>2018/ala/gracenote/summary_gracenote.csv</csv>
 +
 +
====Per-track results====
 +
 +
CW2
 +
<csv>2018/ala/gracenote/CW2_gracenote.csv</csv>
 +
 +
CW3
 +
<csv>2018/ala/gracenote/CW3_gracenote.csv</csv>
 +
 +
GSLW1
 +
<csv>2018/ala/gracenote/GSLW1_gracenote.csv</csv>
 +
 +
GSLW2
 +
<csv>2018/ala/gracenote/GSLW2_gracenote.csv</csv>
 +
 +
GSLW3
 +
<csv>2018/ala/gracenote/GSLW3_gracenote.csv</csv>

Latest revision as of 08:57, 4 November 2018

General Legend

Sub code Submission name Abstract Contributors
CW1 HMM-based Alignment subtask1 [PDF not provided] Chung-Che Wang
CW2 HMM-based Alignment monophone acoustic models subtask2 PDF Chung-Che Wang
CW3 HMM-based Alignment bi-phone acoustic models subtask2 PDF Chung-Che Wang
GSLW1 GMM-HMM (SAT) Models subtask2 PDF Chitralekha Gupta, Bidisha Sharma, Haizhou Li, Ye Wang
GSLW2 SAT+DNN Models subtask2 PDF Chitralekha Gupta, Bidisha Sharma, Haizhou Li, Ye Wang
GSLW3 Non-vocal suppression and SAT Models subtask 2 PDF Chitralekha Gupta, Bidisha Sharma, Haizhou Li, Ye Wang
FFJ1 subtask 1 & 2 [not able to run their code] [PDF not provided] Yang-Bin Dell Fan, Zhe-Cheng Fan, Jyh-Shing Roger Jang

Results

Subtask1: A cappella Mandarin Chinese pop songs

Source-separated

Summary Results

Submission Mean ASE Mean PCS Mean PCETW
CW1 0.12 0.816957532737698 0.938530018742875

download these results as csv

Per-track results

CW1

Track ASE PCS PCETW
cheng_du 0.13 0.796827329414005 0.944591029023747
gang_hao_yu_jian_ni 0.12 0.786115898118654 0.933993399339934
li_ren_chou 0.06 0.847944690265486 0.935483870967742
ping_fan_zhi_lu 0.11 0.816994163344828 0.955752212389381
ti_mian 0.18 0.836905582545517 0.922829581993569

download these results as csv

Mix

Summary Results

Submission Mean ASE Mean PCS Mean PCETW
CW1 3.354 0.363892445941688 0.539771586326105

download these results as csv

Per-track results

CW1

Track ASE PCS PCETW
ke_xi_bu_shi_ni 2.33 0.284673484934437 0.413978494623656
tong_zhuo_de_ni 0.13 0.654883240447694 0.93469387755102
ye_kong_zhong_zui_liang_de_xingxing 10.62 0.08456944596219 0.144117647058824
yu_jian 1.27 0.525205767417589 0.755274261603376
zhui_guang_zhe 2.42 0.270130290946529 0.450793650793651

download these results as csv

Subtask2: English pop songs

Hansen's dataset a cappella

Summary Results

Submission Mean ASE Mean PCS Mean PCETW
CW2 0.35 0.65 0.79
CW3 0.47 0.62 0.81
GSLW1 2.66 0.39 0.48
GSLW2 2.90 0.35 0.40
GSLW3 5.80 0.31 0.31

download these results as csv

Per-track results

CW2

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 0.31 0.570093890373057 0.777372262773723
i_kissed_a_girl 0.2 0.703915182929027 0.848484848484848
clocks_cut 0.19 0.790849995302038 0.835294117647059
rehab 0.17 0.805194335473958 0.865131578947369
rolling_in_the_deep 0.82 0.481665469290785 0.680555555555556
sunrise 0.52 0.513409547058522 0.672
this_afternoon 0.18 0.68360542512406 0.895555555555555
trick_me 0.5 0.530746028428777 0.772413793103448
umbrella 0.52 0.546212118854114 0.72
viva_la_vida 0.13 0.842945012210246 0.879432624113475

download these results as csv

CW3

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 0.32 0.545755317403897 0.744525547445255
i_kissed_a_girl 0.25 0.674245470835476 0.837121212121212
clocks_cut 0.15 0.767154471687807 0.894117647058824
rehab 0.13 0.796814864279409 0.898026315789474
rolling_in_the_deep 0.57 0.522259080218718 0.727777777777778
sunrise 1.72 0.267711468622025 0.504
this_afternoon 0.15 0.683714791923987 0.915555555555556
trick_me 0.86 0.562015684740021 0.875862068965517
umbrella 0.4 0.616793375576461 0.8425
viva_la_vida 0.17 0.78921921137274 0.875886524822695

download these results as csv

GSLW1

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 0.27 0.543259128122919 0.805860805860806
i_kissed_a_girl 0.11 0.763207570129874 0.946969696969697
clocks_cut 0.63 0.554917222456744 0.905882352941176
rehab 2.07 0.071219687000756 0.05921052631579
rolling_in_the_deep 3.15 0.009252640391624 0.027777777777778
sunrise 3.68 0.127399217474657 0.08
this_afternoon 13.61 0 0
trick_me 2.97 0.080867269977835 0
umbrella 0.08 0.795244553082193 0.97323600973236
viva_la_vida 0.04 0.926735844657678 0.98936170212766

download these results as csv

GSLW2

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 0.48 0.47449184033124 0.684981684981685
i_kissed_a_girl 0.34 0.647207424371483 0.829545454545455
clocks_cut 2.33 0.570135233602421 0.717647058823529
rehab 0.85 0.144820599809062 0.006578947368421
rolling_in_the_deep 3.19 0.016695413804996 0.013888888888889
sunrise 4.44 0.041273457518474 0.008
this_afternoon 13.66 0.006072269630639 0
trick_me 2.98 0.058593676899486 0.006896551724138
umbrella 0.48 0.788270711351441 0.788321167883212
viva_la_vida 0.2 0.766924833336262 0.968085106382979

download these results as csv

GSLW3

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 19.85 0.002186924613191 0
i_kissed_a_girl 7.61 0.026256826598641 0.035294117647059
clocks_cut 0.85 0.144820599809062 0.006578947368421
rehab 9.5 0.010902142506288 0.013888888888889
rolling_in_the_deep 4 0.186314997894924 0
sunrise 13.06 0.028738675715158 0.057777777777778
this_afternoon 2.97 0.080867269977835 0
trick_me 0.08 0.795244553082193 0.97323600973236
umbrella 0.04 0.926735844657678 0.98936170212766
viva_la_vida 0.04 0.926735844658 0.99

download these results as csv


Hansen's dataset mix

Summary Results

Submission Mean ASE Mean PCS Mean PCETW
CW2 3.566 0.37802036168513 0.567124924061636
CW3 2.07 0.449797259112808 0.699532109376408
GSLW1 7.01 0.25 0.43
GSLW2 13.32 0.09 0.16
GSLW3 17.78 0.02 0.02

download these results as csv

Per-track results

CW2

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 10.99 0.226124698582149 0.416058394160584
clocks_cut 3.26 0.38930810261648 0.740740740740741
i_kissed_a_girl 0.27 0.621119507879228 0.772727272727273
rehab 0.62 0.530159948664057 0.61437908496732
rolling_in_the_deep 0.69 0.475431465929193 0.694444444444444
sunrise 1.61 0.320656199160953 0.456
this_afternoon 2.31 0.407519185676804 0.61283185840708
trick_me 6.22 0.190157715996446 0.351724137931035
umbrella 8.55 0.179398573857059 0.288939051918736
viva_la_vida 1.14 0.440328218488935 0.723404255319149

download these results as csv

CW3

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 8.2 0.311763886339647 0.58029197080292
clocks_cut 1.94 0.41005479556285 0.802469135802469
i_kissed_a_girl 0.28 0.649341636045947 0.867424242424242
rehab 0.24 0.637445516497089 0.797385620915033
rolling_in_the_deep 0.48 0.500342720534644 0.733333333333333
sunrise 1.04 0.379106004445562 0.68
this_afternoon 2.22 0.469125484189428 0.694690265486726
trick_me 2.38 0.27435310698658 0.520689655172414
umbrella 1.83 0.444926153029708 0.652370203160271
viva_la_vida 2.09 0.421513287496619 0.666666666666667

download these results as csv

GSLW1

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 14.68 0.141693609781414 0.283088235294118
clocks_cut 2.55 0.414053417584349 0.790123456790123
i_kissed_a_girl 0.58 0.503784408057355 0.727272727272727
rehab 1.76 0.27223584836753 0.42156862745098
rolling_in_the_deep 1.82 0.414947059854735 0.619444444444444
sunrise 2.57 0.289407480532988 0.56
this_afternoon 7.4 0.020940117418009 0.019911504424779
trick_me 2.08 0.329192639487664 0.675862068965517
umbrella 6.18 0.001593684168864 0
viva_la_vida 30.47 0.065665086051737 0.163120567375887

download these results as csv

GSLW2

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 28.59 0.036920580858665 0.040441176470588
clocks_cut 16.57 0.187578305002503 0.185185185185185
i_kissed_a_girl 13.51 0.111778055591569 0.181818181818182
rehab 3.67 0.110359041696525 0.225490196078431
rolling_in_the_deep 16.8 0.05627968058309 0.069444444444445
sunrise 3.25 0.214156673851977 0.384
this_afternoon 7.41 0.020940117418009 0.019911504424779
trick_me 4.15 0.177744909245817 0.448275862068966
umbrella 10.44 0.012630023232327 0.029197080291971
viva_la_vida 28.79 0.017583635407709 0.024822695035461

download these results as csv

GSLW3

Track ASE PCS PCETW
Madonna-Beautiful_Stranger_full_ending 29.65 0 0
clocks_cut 7.7 0.077433469687045 0.037037037037037
i_kissed_a_girl 19.85 0.002186924613191 0
rehab 6.5 0.060910986687061 0.058823529411765
rolling_in_the_deep 12.37 0.010704204247576 0.011111111111111
sunrise 31.52 0 0
this_afternoon 7.41 0.020940117418009 0.019911504424779
trick_me 44.46 0.002999541552481 0
umbrella 6.62 0 0
viva_la_vida 11.72 0.029197955750477 0.060283687943263

download these results as csv

Mauch's dataset

Summary Results

Submission Mean ASE Mean PCS Mean PCETW
CW2 6.9205 0.250957778629366 0.367059394634144
CW3 4.128 0.353908343655694 0.515559781202695
GSLW1 22.68 0.13 0.20
GSLW2 24.00 0.05 0.08
GSLW3 23.48 0.02 0.02

download these results as csv

Per-track results

CW2

Track ASE PCS PCETW
Abba.KnowingMeKnowingYou 2.55 0.303434126561623 0.542105263157895
Bangles.EternalFlame 15.75 0.125825535697715 0.161538461538462
Blondie.CallMe 9.92 0.088918751741438 0.151960784313725
Duffy.WarwickAvenue 0.66 0.510263538699712 0.685606060606061
DuranDuran.OrdinaryWorld 3.5 0.407677829323225 0.505747126436782
FranzFerdinand.DoYouWantTo 4.66 0.1659855682221 0.28960396039604
Martika.ToySoldiers 3.93 0.227487084647727 0.216535433070866
Muse.GuidingLight 9.46 0.236177008484634 0.207207207207207
OtisRedding.TheDockOfTheBay 3.42 0.243927660880857 0.419889502762431
RM-P082 11.99 0.11310835607871 0.261538461538462
RM-P084 2.48 0.328738827914366 0.655487804878049
Queen.WeAreTheChampions 8.61 0.178520190036833 0.183783783783784
RobertPalmer.AddictedToLove 3.69 0.235696453261032 0.3125
Santana.BlackMagicWoman 19.15 0.087050004180567 0.091666666666667
SimonAndGarfunkel.Cecilia 5.92 0.298747054845062 0.385650224215247
TakeThat.BackForGood 0.55 0.485022374652446 0.687830687830688
TinaTurner.WhatsLoveGotToDoWithIt 1.04 0.310493123746203 0.513931888544892
Toto.Africa 3.94 0.237180335184384 0.464150943396226
U2.WithOrWithoutYou 25.92 0.107310010826156 0.115183246073298
Zweieck.She 1.27 0.32759173760254 0.489270386266094

download these results as csv

CW3

Track ASE PCS PCETW
Abba.KnowingMeKnowingYou 1.24 0.372548439857921 0.736842105263158
Bangles.EternalFlame 2.36 0.355346819343771 0.457692307692308
Blondie.CallMe 7.98 0.232911906837266 0.377450980392157
Duffy.WarwickAvenue 0.58 0.636252312551692 0.810606060606061
DuranDuran.OrdinaryWorld 3.71 0.371654237318203 0.590038314176245
FranzFerdinand.DoYouWantTo 2.02 0.221245810522692 0.331683168316832
Martika.ToySoldiers 6.34 0.151566757874127 0.18503937007874
Muse.GuidingLight 5.35 0.275057319635222 0.378378378378378
OtisRedding.TheDockOfTheBay 1.2 0.381426592789676 0.723756906077348
RM-P082 1.49 0.401705052986889 0.68
RM-P082 1.34 0.466228918957039 0.771341463414634
Queen.WeAreTheChampions 3.69 0.343834834099626 0.394594594594595
RobertPalmer.AddictedToLove 1.57 0.406761685287285 0.605113636363636
Santana.BlackMagicWoman 23.3 0.079954805787986 0.066666666666667
SimonAndGarfunkel.Cecilia 2.23 0.382283532174434 0.493273542600897
TakeThat.BackForGood 0.82 0.541557749909341 0.727513227513228
TinaTurner.WhatsLoveGotToDoWithIt 1.82 0.366169083396327 0.544891640866873
Toto.Africa 2.46 0.48609431653068 0.573584905660377
U2.WithOrWithoutYou 12.48 0.148465798488177 0.214659685863874
Zweieck.She 0.58 0.457100898765531 0.648068669527897

download these results as csv

GSLW1

Track ASE PCS PCETW
Abba.KnowingMeKnowingYou 16.88 0.013470170808295 0
Bangles.EternalFlame 15.94 0.095904155405214 0.133333333333333
Blondie.CallMe 28.57 0.002655582320378 0.009803921568627
Duffy.WarwickAvenue 2.16 0.446066658042375 0.59469696969697
DuranDuran.OrdinaryWorld 84.8 0.004470062072001 0.007662835249042
FranzFerdinand-DoYouWanTo 14.81 0.00123317780798 0
Martika.ToySoldiers 22.85 0.0472493846477 0.08
Muse.GuidingLight 53.23 0.002459670343282 0
OtisRedding.TheDockOfTheBay 0.75 0.431955664668173 0.67027027027027
RM-P082 27.19 0.009897495874415 0.024615384615385
RM-P084 3.82 0.268961562762 0.6
Queen.WeAreTheChampions 6.21 0.168930793169075 0.27027027027027
RobertPalmer.AddictedToLove 21.61 0.060521227685247 0.105113636363636
Santana.BlackMagicWoman 30.05 0.007431625327099 0
SimonAndGarfunkel.Cecilia 8.35 0.14738507491501 0.195555555555556
TakeThat.BackForGood 9.66 0.224339838472486 0.301587301587302
TinaTurner.WhatsLoveGotToDoWithIt 1.03 0.384201447282714 0.603715170278638
Toto.Africa 62.87 0.017247645047679 0.026515151515152
U2.WithOrWithoutYou 40.93 0.071640884038462 0.047120418848168
Zweieck.She 1.88 0.225838121314841 0.377682403433476

download these results as csv

GSLW2

Track ASE PCS PCETW
Abba.KnowingMeKnowingYou 16.87 0.014757255231965 0
Bangles.EternalFlame 18.02 0.016713261160298 0.007843137254902
Blondie.CallMe 13.23 0.012906760443102 0.004901960784314
Duffy.WarwickAvenue 17.96 0.048060853575987 0.064393939393939
DuranDuran.OrdinaryWorld 89.63 0.001749418704724 0
FranzFerdinand-DoYouWanTo 13.82 0.00143317780798 0
Martika.ToySoldiers 22.11 0.022312360232 0.07
Muse.GuidingLight 25.23 0.010007061111497 0
OtisRedding.TheDockOfTheBay 8.97 0.164636830770562 0.281081081081081
RM-P082 25.03 0.02169671138624 0.043076923076923
RM-P084 10.44 0.0842236313877 0.18
Queen.WeAreTheChampions 7.45 0.083141205462988 0.102702702702703
RobertPalmer.AddictedToLove 24.85 0.019519136078061 0.025568181818182
Santana.BlackMagicWoman 47.59 0.002782610861555 0
SimonAndGarfunkel.Cecilia 16.48 0.02929144830689 0.035555555555556
TakeThat.BackForGood 8.59 0.13370556161581 0.206349206349206
TinaTurner.WhatsLoveGotToDoWithIt 8.18 0.193299469369439 0.334365325077399
Toto.Africa 48.41 0.069528995862025 0.174242424242424
U2.WithOrWithoutYou 41.09 0.007357461676479 0
Zweieck.She 16.09 0.041447215196855 0.051502145922747

download these results as csv

GSLW3

Track ASE PCS PCETW
Abba.KnowingMeKnowingYou 16.78 0.013470170808295 0
Bangles.EternalFlame 11.14 0.02898885485713 0.027450980392157
Blondie.CallMe 38.38 0 0
Duffy.WarwickAvenue 22.44 0.002410707267485 0.003787878787879
DuranDuran.OrdinaryWorld 26.87 0.026182638371089 0.030651340996169
FranzFerdinand-DoYouWanTo 13.38 0.00161317780798 0
Martika.ToySoldiers 21.94 0.0166667440527 0.03
Muse.GuidingLight 19.4 0.023893393487038 0.008620689655172
OtisRedding.TheDockOfTheBay 41.62 0 0
RM-P082 36.45 0.002038977644714 0
RM-P084 9.48 0.08661632622508 0.2
Queen.WeAreTheChampions 26.9 0 0
RobertPalmer.AddictedToLove 31.52 0 0
Santana.BlackMagicWoman 29.5 0.123035938934082 0
SimonAndGarfunkel.Cecilia 38.23 0 0
TakeThat.BackForGood 10.99 0.028435153463603 0.047619047619048
TinaTurner.WhatsLoveGotToDoWithIt 25.93 0.02372512809696 0.015479876160991
Toto.Africa 11.81 0.002388174061774 0.018939393939394
U2.WithOrWithoutYou 14.81 0.011272516767769 0.005235602094241
Zweieck.She 22.05 0.00600489576593 0.008583690987124

download these results as csv


Gracenote dataset

Summary Results

Submission Mean ASE Mean PCS Mean PCETW
CW2 0.7025 0.458735456325263 0.60906181930376
CW3 2.33375 0.511205139349735 0.643623014839236
GSLW1 6.60 0.37 0.44
GSLW2 4.72 0.19 0.24
GSLW3 52.10 0.00 0.00

download these results as csv

Per-track results

CW2

Track ASE PCS PCETW
1979 2.08 0.189524289551553 0.340909090909091
abba 0.98 0.465395243109086 0.452054794520548
biggie 0.81 0.329478528034905 0.474789915966386
daft_punk 0.46 0.423693600833937 0.7
hand 0.48 0.510502645779476 0.666666666666667
jackson 0.21 0.521940486426134 0.725663716814159
metalica 0.31 0.713282043872111 0.836734693877551
swift 0.29 0.516066812994904 0.675675675675676

download these results as csv

CW3

Track ASE PCS PCETW
1979 16.17 0.002083709800278 0
abba 0.74 0.574175520019131 0.575342465753425
biggie 0.67 0.396017715374031 0.592436974789916
daft_punk 0.24 0.530720324881901 0.790909090909091
hand 0.16 0.630913612888076 0.80952380952381
jackson 0.25 0.55487554898221 0.734513274336283
metalica 0.05 0.881932715921628 0.979591836734694
swift 0.39 0.518921966930623 0.666666666666667

download these results as csv

GSLW1

Track ASE PCS PCETW
1979 0.7 0.471736541841666 0.477272727272727
abba 34.81 0 0
biggie 0.1 0.710141498785837 0.899159663865546
daft_punk 3.45 0.014179411316212 0
hand 0.43 0.6370494418781 0.73015873015873
jackson 0.41 0.458996116408416 0.592920353982301
metalica 11.91 0.112442602544362 0.122448979591837
swift 0.98 0.555695178426554 0.702702702702703

download these results as csv

GSLW2

Track ASE PCS PCETW
1979 12.9 0.046342268559824 0.045454545454546
abba 7.08 0.032851312176306 0
biggie 0.26 0.57316635402032 0.747899159663866
daft_punk 5.02 0.03246320447596 0.063636363636364
hand 1 0.47098537428266 0.555555555555556
jackson 4.03 0.055932716662936 0.097345132743363
metalica 5.93 0.081781803648049 0.061224489795918
swift 1.57 0.243281251760399 0.324324324324324

download these results as csv

GSLW3

Track ASE PCS PCETW
1979 18.13 0 0
abba 44.19 0.008399393011728 0.027397260273973
biggie 74.04 0 0
daft_punk 85.49 0 0
hand 24.49 0 0
jackson 56.51 0 0
metalica 41.98 0 0
swift 71.94 0 0

download these results as csv

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2018:Automatic_Lyrics-to-Audio_Alignment_Results&oldid=12875"