Difference between revisions of "2017:Automatic Lyrics-to-Audio Alignment Results"
From MIREX Wiki
(→Mauchs's Dataset) |
(→Mauchs's Dataset) |
||
| Line 76: | Line 76: | ||
| − | === | + | ===Mauch's Dataset=== |
====Summary Results==== | ====Summary Results==== | ||
Revision as of 03:59, 18 October 2017
Contents
General Legend
| Sub code | Submission name | Abstract | Contributors |
|---|---|---|---|
| AK1 | HMM-based Alignment | Anna Kruspe | |
| AK2 | DTW-based Alignment | Anna Kruspe | |
| AK3 | Levenshtein-based Alignment | Anna Kruspe | |
| DMS1 | Automatic Lyrics-to-Audio Alignment with Harmonic Modeling | Georgi Dzhambazov, Xavier Serra | |
| DMS2 | Automatic Lyrics-to-Audio Alignment with Source Separation | Georgi Dzhambazov, Marius Miron |
Results
Hansen's Dataset A cappella
Summary Results
| Submission | Mean error | Median error | St. dev. error | Mean percentage | Median percentage | St. dev. percentage |
|---|---|---|---|---|---|---|
| AK1 | 5.11 | 0.67 | 12.88 | 0.61 | 0.70 | 0.26 |
| AK2 | 9.77 | 1.82 | 22.62 | 0.46 | 0.39 | 0.27 |
| AK3 | 2.87 | 0.26 | 6.74 | 0.60 | 0.63 | 0.20 |
| DMS1 | 4.46 | 0.37 | 10.71 | 0.55 | 0.69 | 0.27 |
| DMS2 | 4.46 | 0.37 | 10.71 | 0.55 | 0.69 | 0.27 |
Per-song results
AK1
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| clocks | 41.510 | 0.008 |
| i_kissed_a_girl | 0.180 | 0.721 |
| rehab | 0.040 | 0.927 |
| rolling_in_the_deep | 0.860 | 0.438 |
| sunrise | 0.670 | 0.700 |
| this_afternoon | 0.990 | 0.599 |
| trick_me | 0.170 | 0.751 |
| umbrella | 1.450 | 0.462 |
| viva_la_vida | 0.120 | 0.842 |
AK2
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| clocks | 73.560 | 0.008 |
| i_kissed_a_girl | 0.270 | 0.626 |
| rehab | 0.290 | 0.795 |
| rolling_in_the_deep | 3.930 | 0.284 |
| sunrise | 0.470 | 0.761 |
| this_afternoon | 1.820 | 0.385 |
| trick_me | 0.110 | 0.804 |
| umbrella | 2.040 | 0.297 |
| viva_la_vida | 5.470 | 0.209 |
AK3
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| clocks | 21.890 | 0.152 |
| i_kissed_a_girl | 0.140 | 0.704 |
| rehab | 0.070 | 0.889 |
| rolling_in_the_deep | 1.060 | 0.489 |
| sunrise | 0.760 | 0.627 |
| this_afternoon | 0.260 | 0.545 |
| trick_me | 0.140 | 0.762 |
| umbrella | 1.300 | 0.489 |
| viva_la_vida | 0.210 | 0.742 |
DMS1
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| clocks | 34.710 | 0.007 |
| i_kissed_a_girl | 0.160 | 0.727 |
| rehab | 0.110 | 0.876 |
| rolling_in_the_deep | 1.220 | 0.350 |
| sunrise | 0.360 | 0.786 |
| this_afternoon | 1.320 | 0.438 |
| trick_me | 0.190 | 0.720 |
| umbrella | 1.720 | 0.333 |
| viva_la_vida | 0.370 | 0.686 |
DMS2
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| clocks | 34.710 | 0.007 |
| i_kissed_a_girl | 0.160 | 0.727 |
| rehab | 0.110 | 0.876 |
| rolling_in_the_deep | 1.220 | 0.350 |
| sunrise | 0.360 | 0.786 |
| this_afternoon | 1.320 | 0.438 |
| trick_me | 0.190 | 0.720 |
| umbrella | 1.720 | 0.333 |
| viva_la_vida | 0.370 | 0.686 |
Hansen's Dataset
Summary Results
| Submission | Mean error | Median error | St. dev. error | Mean percentage | Median percentage | St. dev. percentage |
|---|---|---|---|---|---|---|
| AK1 | 13.96 | 7.64 | 12.73 | 0.12 | 0.06 | 0.12 |
| AK2 | 27.94 | 19.58 | 31.86 | 0.10 | 0.03 | 0.13 |
| AK3 | 7.34 | 4.55 | 7.71 | 0.25 | 0.28 | 0.15 |
| DMS1 | 26.70 | 14.94 | 29.97 | 0.04 | 0.02 | 0.05 |
| DMS2 | 10.57 | 2.02 | 23.30 | 0.34 | 0.44 | 0.18 |
Per-song results
AK1
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| clocks | 42.760 | 0.002 |
| i_kissed_a_girl | 7.640 | 0.052 |
| rehab | 6.040 | 0.061 |
| rolling_in_the_deep | 2.990 | 0.245 |
| sunrise | 3.240 | 0.308 |
| this_afternoon | 2.650 | 0.277 |
| trick_me | 18.130 | 0.078 |
| umbrella | 16.330 | 0.014 |
| viva_la_vida | 25.820 | 0.009 |
AK2
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| clocks | 107.720 | 0.001 |
| i_kissed_a_girl | 6.590 | 0.067 |
| rehab | 5.780 | 0.164 |
| rolling_in_the_deep | 31.660 | 0.002 |
| sunrise | 4.260 | 0.405 |
| this_afternoon | 24.200 | 0.029 |
| trick_me | 1.800 | 0.231 |
| umbrella | 49.870 | 0.004 |
| viva_la_vida | 19.580 | 0.017 |
DMS1
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| clocks | 65.430 | 0.001 |
| i_kissed_a_girl | 5.720 | 0.094 |
| rehab | 5.660 | 0.060 |
| rolling_in_the_deep | 19.390 | 0.005 |
| sunrise | 14.940 | 0.133 |
| this_afternoon | 17.310 | 0.008 |
| trick_me | 9.910 | 0.018 |
| umbrella | 95.470 | 0.000 |
| viva_la_vida | 6.440 | 0.070 |
DMS2
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| clocks | 76.200 | 0.010 |
| i_kissed_a_girl | 1.460 | 0.443 |
| rehab | 0.550 | 0.532 |
| rolling_in_the_deep | 7.530 | 0.192 |
| sunrise | 2.020 | 0.501 |
| this_afternoon | 0.700 | 0.481 |
| trick_me | 0.390 | 0.505 |
| umbrella | 2.650 | 0.189 |
| viva_la_vida | 3.630 | 0.212 |
Mauch's Dataset
Summary Results
| Submission | Mean error | Median error | St. dev. error | Mean percentage | Median percentage | St. dev. percentage |
|---|---|---|---|---|---|---|
| AK1 | 17.70 | 10.14 | 17.01 | 0.08 | 0.04 | 0.10 |
| AK2 | 22.23 | 16.38 | 22.08 | 0.02 | 0.00 | 0.04 |
| AK3 | 9.03 | 7.52 | 7.23 | 0.15 | 0.15 | 0.10 |
| DMS1 | 14.91 | 10.14 | 15.30 | 0.04 | 0.03 | 0.04 |
| DMS2 | 11.64 | 5.12 | 13.31 | 0.14 | 0.14 | 0.10 |
Per-song results
AK1
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| Abba.KnowingMeKnowingYou | 9.400 | 0.064 |
| Bangles.EternalFlame | 38.270 | 0.007 |
| Blondie.CallMe | 34.340 | 0.011 |
| Duffy.WarwickAvenue | 7.200 | 0.176 |
| DuranDuran.OrdinaryWorld | 12.780 | 0.020 |
| FranzFerdinand.DoYouWantTo | 5.520 | 0.046 |
| Martika.ToySoldiers | 20.050 | 0.001 |
| Muse.GuidingLight | 66.170 | 0.002 |
| OtisRedding.TheDockOfTheBay | 0.920 | 0.348 |
| Queen.WeAreTheChampions | 6.430 | 0.088 |
| RM-P082 | 43.130 | 0.034 |
| RM-P084 | 9.520 | 0.101 |
| RobertPalmer.AddictedToLove | 10.750 | 0.019 |
| Santana.BlackMagicWoman | 39.870 | 0.004 |
| SimonAndGarfunkel.Cecilia | 5.940 | 0.041 |
| TakeThat.BackForGood | 3.090 | 0.125 |
| TinaTurner.WhatsLoveGotToDoWithIt | 10.850 | 0.113 |
| Toto.Africa | 4.650 | 0.299 |
| U2.WithOrWithoutYou | 19.570 | 0.024 |
| Zweieck.She | 5.610 | 0.174 |
AK2
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| Abba.KnowingMeKnowingYou | 4.250 | 0.052 |
| Bangles.EternalFlame | 6.680 | 0.021 |
| Blondie.CallMe | 5.980 | 0.035 |
| Duffy.WarwickAvenue | 16.040 | 0.051 |
| DuranDuran.OrdinaryWorld | 53.460 | 0.003 |
| FranzFerdinand.DoYouWantTo | 18.410 | 0.001 |
| Martika.ToySoldiers | 30.370 | 0.001 |
| Muse.GuidingLight | 94.140 | 0.000 |
| OtisRedding.TheDockOfTheBay | 1.380 | 0.145 |
| Queen.WeAreTheChampions | 18.500 | 0.018 |
| RM-P082 | 14.400 | 0.002 |
| RM-P084 | 12.170 | 0.004 |
| RobertPalmer.AddictedToLove | 20.840 | 0.001 |
| Santana.BlackMagicWoman | 16.720 | 0.003 |
| SimonAndGarfunkel.Cecilia | 7.430 | 0.004 |
| TakeThat.BackForGood | 3.930 | 0.045 |
| TinaTurner.WhatsLoveGotToDoWithIt | 6.860 | 0.085 |
| Toto.Africa | 41.390 | 0.001 |
| U2.WithOrWithoutYou | 51.360 | 0.000 |
| Zweieck.She | 20.240 | 0.000 |
DMS1
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| Abba.KnowingMeKnowingYou | 18.910 | 0.003 |
| Bangles.EternalFlame | 4.010 | 0.079 |
| Blondie.CallMe | 8.110 | 0.062 |
| Duffy.WarwickAvenue | 3.350 | 0.154 |
| DuranDuran.OrdinaryWorld | 6.250 | 0.077 |
| FranzFerdinand.DoYouWantTo | 5.280 | 0.031 |
| Martika.ToySoldiers | 11.150 | 0.015 |
| Muse.GuidingLight | 35.620 | 0.030 |
| OtisRedding.TheDockOfTheBay | 3.660 | 0.062 |
| Queen.WeAreTheChampions | 9.120 | 0.020 |
| RM-P082 | 12.430 | 0.009 |
| RM-P084 | 15.190 | 0.022 |
| RobertPalmer.AddictedToLove | 8.320 | 0.014 |
| Santana.BlackMagicWoman | 21.840 | 0.013 |
| SimonAndGarfunkel.Cecilia | 5.180 | 0.065 |
| TakeThat.BackForGood | 4.130 | 0.031 |
| TinaTurner.WhatsLoveGotToDoWithIt | 30.020 | 0.000 |
| Toto.Africa | 12.420 | 0.053 |
| U2.WithOrWithoutYou | 70.160 | 0.000 |
| Zweieck.She | 13.050 | 0.029 |
DMS2
| Track | Average absolute error | Percentage of correct segments |
|---|---|---|
| Abba.KnowingMeKnowingYou | 4.310 | 0.206 |
| Bangles.EternalFlame | 3.080 | 0.140 |
| Blondie.CallMe | 4.080 | 0.140 |
| Duffy.WarwickAvenue | 9.400 | 0.210 |
| DuranDuran.OrdinaryWorld | 4.320 | 0.123 |
| FranzFerdinand.DoYouWantTo | 42.670 | 0.000 |
| Martika.ToySoldiers | 17.600 | 0.037 |
| Muse.GuidingLight | 48.800 | 0.000 |
| OtisRedding.TheDockOfTheBay | 0.380 | 0.374 |
| Queen.WeAreTheChampions | 5.430 | 0.219 |
| RM-P082 | 12.210 | 0.049 |
| RM-P084 | 1.510 | 0.242 |
| RobertPalmer.AddictedToLove | 4.820 | 0.057 |
| Santana.BlackMagicWoman | 16.830 | 0.056 |
| SimonAndGarfunkel.Cecilia | 29.230 | 0.021 |
| TakeThat.BackForGood | 1.730 | 0.172 |
| TinaTurner.WhatsLoveGotToDoWithIt | 9.700 | 0.147 |
| Toto.Africa | 2.390 | 0.268 |
| U2.WithOrWithoutYou | 11.790 | 0.102 |
| Zweieck.She | 2.480 | 0.197 |
