Difference between revisions of "2015:Singing Voice Separation Results"
From MIREX Wiki
(→Description) |
|||
| (One intermediate revision by the same user not shown) | |||
| Line 3: | Line 3: | ||
=== Description === | === Description === | ||
| − | These are the results for the 2015 running of the Singing Voice Separation task set. For more information about this task set please refer to the [[2015:Singing Voice Separation]] page. | + | These are the results for the 2015 running of the Singing Voice Separation task set. The evaluation set is kindly provided by [http://mac.citi.sinica.edu.tw/ikala/ iKala]. If you need to cite this page, please also cite T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722. For more information about this task set please refer to the [[2015:Singing Voice Separation]] page. |
=== Legend === | === Legend === | ||
| Line 21: | Line 21: | ||
|- | |- | ||
! IIY3 | ! IIY3 | ||
| − | | | + | | MIREX2015: Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/IIY3.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii |
|- | |- | ||
! IIY4 | ! IIY4 | ||
| − | | | + | | MIREX2015: Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/IIY4.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii |
|- | |- | ||
! MD3 | ! MD3 | ||
Latest revision as of 02:42, 3 August 2016
Contents
Introduction
Description
These are the results for the 2015 running of the Singing Voice Separation task set. The evaluation set is kindly provided by iKala. If you need to cite this page, please also cite T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722. For more information about this task set please refer to the 2015:Singing Voice Separation page.
Legend
| Submission code | Submission name | Abstract PDF | Contributors |
|---|---|---|---|
| FJ1 | MIREX 2015 Submission for Singing Voice Separation | - | Zhe-Cheng Fan, Jyh-Shing Roger Jang |
| FJ2 | MIREX 2015 Submission for Singing Voice Separation | - | Zhe-Cheng Fan, Jyh-Shing Roger Jang |
| IIY3 | MIREX2015: Singing Voice Separation | Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii | |
| IIY4 | MIREX2015: Singing Voice Separation | Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii | |
| MD3 | An Ensemble Method for Learning to Extract Vocals from Polyphonic Musical Audio | Matt McVicar, Tijl De Bie | |
| MD4 | An Ensemble Method for Learning to Extract Vocals from Polyphonic Musical Audio | Matt McVicar, Tijl De Bie |
Evaluation Criteria
GNSDR = Global Normalized Signal-to-Distortion Ratio
NSDR = Normalized Signal-to-Distortion Ratio
SIR = Signal-to-Interference Ratio
SAR = Signal-to-Artifacts Ratio
Summary
Summary Results
| Algorithm | Voice GNSDR (dB) | Music GNSDR (dB) | Runtime (m) |
|---|---|---|---|
| FJ1 | 6.8236 | 10.135 | 3.4014 |
| FJ2 | 6.3487 | 9.8678 | 2.9135 |
| IIY3 | 4.9862 | 8.2138 | 99.1737 |
| IIY4 | 5.3953 | 8.77 | 46.6621 |
| MD3 | 2.9831 | 6.3671 | 121.6655 |
| MD4 | 3.1022 | 7.4657 | 121.1348 |
NSDR
For the Singing Voice (dB)
| Algorithm | Mean | SD | Min | Max | Median |
|---|---|---|---|---|---|
| FJ1 | 6.8236 | 3.3945 | -1.0987 | 17.087 | 6.693 |
| FJ2 | 6.3487 | 3.4062 | -1.5522 | 16.139 | 6.3066 |
| IIY3 | 4.9862 | 3.0974 | -1.4384 | 15.753 | 4.5152 |
| IIY4 | 5.3953 | 3.1454 | -1.5708 | 16.296 | 5.321 |
| MD3 | 2.9831 | 3.359 | -3.6909 | 14.99 | 2.991 |
| MD4 | 3.1022 | 2.3219 | -1.5255 | 10.167 | 3.082 |
For the Music Accompaniment (dB)
| Algorithm | Mean | SD | Min | Max | Median |
|---|---|---|---|---|---|
| FJ1 | 10.135 | 2.9105 | 2.2446 | 17.147 | 10.139 |
| FJ2 | 9.8678 | 2.7308 | 3.379 | 15.962 | 9.6999 |
| IIY3 | 8.2138 | 3.5171 | -11.987 | 14.139 | 8.8945 |
| IIY4 | 8.77 | 3.8443 | -14.661 | 16.429 | 9.2513 |
| MD3 | 6.3671 | 3.0928 | -1.2317 | 14.238 | 6.3533 |
| MD4 | 7.4657 | 3.8335 | -7.6267 | 17.156 | 7.6152 |
Boxplots
SIR
For the Singing Voice (dB)
| Algorithm | Mean | SD | Min | Max | Median |
|---|---|---|---|---|---|
| FJ1 | 13.346 | 10.351 | -23.786 | 37.811 | 15.755 |
| FJ2 | 13.746 | 10.18 | -23.665 | 33.926 | 16.683 |
| IIY3 | 15.717 | 11.713 | -27.859 | 32.436 | 18.944 |
| IIY4 | 14.626 | 11.323 | -28.429 | 31.364 | 16.947 |
| MD3 | 11.361 | 10.162 | -24.183 | 25.137 | 14.065 |
| MD4 | 6.6966 | 9.8508 | -30.777 | 19.816 | 9.1638 |
For the Music Accompaniment (dB)
| Algorithm | Mean | SD | Min | Max | Median |
|---|---|---|---|---|---|
| FJ1 | 11.203 | 8.9932 | -2.3519 | 43.1 | 9.5306 |
| FJ2 | 11.653 | 9.2526 | -4.0102 | 44.696 | 9.6655 |
| IIY3 | 13.288 | 8.3022 | 1.3372 | 40.385 | 12.376 |
| IIY4 | 15.057 | 8.4537 | 2.1673 | 43.782 | 14.952 |
| MD3 | 9.7413 | 9.8699 | -5.4239 | 46.37 | 8.2292 |
| MD4 | 16.616 | 8.4294 | 1.1789 | 41.702 | 15.186 |
Boxplots
SAR
For the Singing Voice (dB)
| Algorithm | Mean | SD | Min | Max | Median |
|---|---|---|---|---|---|
| FJ1 | 11.517 | 8.3821 | -22.529 | 20.922 | 13.88 |
| FJ2 | 10.586 | 8.1887 | -23.897 | 17.987 | 13.214 |
| IIY3 | 8.5224 | 7.4896 | -25.194 | 17.422 | 10.527 |
| IIY4 | 9.214 | 7.6286 | -24.313 | 17.835 | 11.522 |
| MD3 | 7.1543 | 6.743 | -19.385 | 19.311 | 8.2378 |
| MD4 | 11.651 | 7.329 | -13.815 | 27.014 | 12.629 |
For the Music Accompaniment (dB)
| Algorithm | Mean | SD | Min | Max | Median |
|---|---|---|---|---|---|
| FJ1 | 9.9624 | 3.2432 | -0.75164 | 18.359 | 10.182 |
| FJ2 | 9.082 | 3.2594 | 0.040332 | 17.077 | 9.1409 |
| IIY3 | 5.754 | 3.3033 | -4.0821 | 16.133 | 5.4503 |
| IIY4 | 5.9472 | 3.4381 | -6.7539 | 16.601 | 5.9557 |
| MD3 | 5.3175 | 3.8621 | -6.6715 | 15.684 | 5.324 |
| MD4 | 4.2005 | 3.5672 | -5.4033 | 11.8 | 4.6933 |
Boxplots
Runtime Data
| Submission Code | Runtime (m) |
|---|---|
| FJ1 | 3.4014 |
| FJ2 | 2.9135 |
| IIY3 | 99.1737 |
| IIY4 | 46.6621 |
| MD3 | 121.6655 |
| MD4 | 121.1348 |
