MIREX Wiki - User contributions [en]

User:Tak-Shing Chan

2017-06-21T11:00:08Z

Tak-Shing Chan: Reverted edits by Tak-Shing Chan (talk) to last revision by Kahyun Choi

Tak-Shing Chan received his PhD from the University of London. In 2011, he worked as a Research Associate at the Hong Kong Polytechnic University. He is currently a Postdoctoral Fellow at the Academia Sinica. His research interests include sparse coding, signal processing, music cognition, and distributed systems.

User:Tak-Shing Chan

2017-06-21T10:56:37Z

Tak-Shing Chan:

Tak-Shing T. Chan received the Ph.D. degree in computing from the University of London in 2008. From 2006 to 2008, he was a Scientific Programmer at the University of Sheffield. In 2011, he worked as a Research Associate at the Hong Kong Polytechnic University. He is currently a Postdoctoral Fellow at Academia Sinica, Taiwan.

User:Tak-Shing Chan

2017-06-21T10:56:14Z

Tak-Shing Chan:

Tak-Shing T. Chan received the Ph.D. degree in cmoputing from the University of London in 2008. From 2006 to 2008, he was a Scientific Programmer at the University of Sheffield. In 2011, he worked as a Research Associate at the Hong Kong Polytechnic University. He is currently a Postdoctoral Fellow at Academia Sinica, Taiwan.

2016:Singing Voice Separation

2016-08-04T05:09:12Z

Tak-Shing Chan: /* Data */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms (these are the hidden parts of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset). If your algorithm is a supervised one, you are welcome to use the public part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset for training. In addition, you can train with 30-second segments from the SiSEC [https://sisec.inria.fr/home/2016-professionally-produced-music-recordings/ MUS] challenge.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

For more information about the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset, see T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722.

Remark. The hidden parts of the iKala dataset are quite similar to the public set, but the training set from SiSEC is quite different.

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0] (but with the permutation part removed, as the ability to classify signals is part of the singing voice separation challenge). Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

Remark. You may assume that trueMixed is always in the range of [-1,1].

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2015 Submission Instructions below). For supervised submissions, please provide training details in the extended abstract (e.g. datasets used).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2016 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2016:Singing Voice Separation

2016-08-04T04:50:09Z

Tak-Shing Chan: /* Evaluation */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms (these are the hidden parts of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset). If your algorithm is a supervised one, you are welcome to use the public part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset for training. In addition, you can train with 30-second segments from the SiSEC [https://sisec.inria.fr/home/2016-professionally-produced-music-recordings/ MUS] challenge.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

For more information about the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset, see T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722.

This is a comment by Marius Miron: how is it different the hidden dataset from the known one. Supervised approaches need to know this in order to know which transformations should include into the training part. Does it have different tempo, voices, timbres, genre, audio amplitudes? What are the factors that you need to make your algorithm robust to?

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0] (but with the permutation part removed, as the ability to classify signals is part of the singing voice separation challenge). Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

Remark. You may assume that trueMixed is always in the range of [-1,1].

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2015 Submission Instructions below). For supervised submissions, please provide training details in the extended abstract (e.g. datasets used).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2016 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2016:Singing Voice Separation Results

2016-08-03T08:56:21Z

Tak-Shing Chan: /* Legend */

== Introduction ==

=== Description ===

These are the results for the 2016 running of the Singing Voice Separation task set. The evaluation set is kindly provided by [http://mac.citi.sinica.edu.tw/ikala/ iKala]. If you need to cite this page, please also cite T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722. For more information about this task set please refer to the [[2016:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
! width="80" | Training set
|-
! GD1
| Harmonic Modeling of Singing Voice for Source Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/GD1.pdf PDF] || Georgi Dzhambazov || Unknown
|-
! HC1
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/HC1.pdf PDF] || Yi-Chun Huang, Tai-Shih Chi || iKala
|-
! LCP1
| Deep Clustering for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/LCP1.pdf PDF] || Yi Luo, Zhuo Chen, John R. Hershey, Jonathan Le Roux, Daniel P. W. Ellis || SiSEC
|-
! LCP2
| Deep Clustering for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/LCP2.pdf PDF] || Yi Luo, Zhuo Chen, John R. Hershey, Jonathan Le Roux, Daniel P. W. Ellis || SiSEC
|-
! MC2
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | - || Marius Miron, Pritish Chandna || Unknown
|-
! MC3
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | - || Marius Miron, Pritish Chandna || Unknown
|-
! RSGP1
| Singing Voice Separation Using Deep Neural Networks and F0 Estimation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/RSGP1.pdf PDF] || Gerard Roma, Emad M. Grais, Andrew J. R. Simpson, Mark D. Plumbley || iKala
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (m)
|-
| GD1 || -2.2810 || 0.3954 || 26.4413
|-
| HC1 || 4.6309 || 7.8180 || 28.9727
|-
| LCP1 || 6.0726 || 10.9256 || 37.8235
|-
| LCP2 || 6.3414 || 11.1878 || 32.4800
|-
| MC2 || 5.2891 || 9.6678 || 34.8084
|-
| MC3 || 5.4920 || 9.8049 || 36.7194
|-
| RSGP1 || 3.2589 || 8.7664 || 32.3578
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-sar.png]]

== Runtime Data ==

<csv>2016/svs/runtime.csv</csv>

2014:Singing Voice Separation Results

2016-08-03T08:42:54Z

Tak-Shing Chan: /* Description */

== Introduction ==

=== Description ===

These are the results for the 2014 running of the Singing Voice Separation task set. The evaluation set is kindly provided by [http://mac.citi.sinica.edu.tw/ikala/ iKala]. If you need to cite this page, please also cite T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722. For more information about this task set please refer to the [[2014:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! GW1
| Bayesian Singing-Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/GW1.pdf PDF] || Guan-Xiang Wang, Po-Kai Yang, Chung-Chien Hsu, Jen-Tzung Chien
|-
! HKHS1
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS1.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! HKHS2
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS2.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! HKHS3
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS3.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! IIY1
| Singing Voice Separation and Vocal F0 Estimation based on Robust PCA and Subharmonic Summation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/IIY1.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! IIY2
| Singing Voice Separation and Vocal F0 Estimation based on Robust PCA and Subharmonic Summation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/IIY2.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! JL1
| Singing Voice Separation Based on Sparse Nature and Spectral/Temporal Discontinuity || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/JL1.pdf PDF] || Il-Young Jeong, Kyogu Lee
|-
! LFR1
| Kernel Additive Modelling with light models || style="text-align: center;" | [http://dx.doi.org/10.1109/ICASSP.2015.7177935 PDF] || Antoine Liutkus, Derry Fitzgerald, Zafar Rafii
|-
! RNA1
| Singing Voice Separation using Adaptive Window Harmonic Sinusoidal Modeling || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/RNA1.pdf PDF] || Preeti Rao, Nagesh Nayak, Sharath Adavanne
|-
! RP1
| REPET-SIM for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/RP1.pdf PDF] || Zafar Rafii, Bryan Pardo
|-
! YC1
| MIREX 2014 Submission for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/YC1.pdf PDF] || Frederick Yen, Tai-Shih Chi
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (hh)
|-
| GW1 || 2.8861 || 5.2549 || 24
|-
| HKHS1 || -1.3988 || 0.3483 || 06
|-
| HKHS2 || -1.9413 || 0.5239 || 06
|-
| HKHS3 || -2.4807 || 0.1414 || 06
|-
| IIY1 || 4.2190 || 7.7893 || 02
|-
| IIY2 || 4.4764 || 7.8661 || 02
|-
| JL1 || 4.1564 || 5.6304 || 01
|-
| LFR1 || 0.6499 || 3.0867 || 03
|-
| RNA1 || 3.6915 || 7.3153 || 06
|-
| RP1 || 2.8602 || 5.0306 || 01
|-
| YC1 || -0.8202 || -3.1150 || 13
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-sar.png]]

== Individual Spectrograms ==

As the MIREX test set is private, we use three other songs with similar characteristics to demonstrate the algorithms.

{| border="1" cellspacing="0" style="text-align: left;"
|-
| [[File:2014-svs-gw1.png|thumb|Spectrograms for GW1]]
| [[File:2014-svs-hkhs1.png|thumb|Spectrograms for HKHS1]]
| [[File:2014-svs-hkhs2.png|thumb|Spectrograms for HKHS2]]
| [[File:2014-svs-hkhs3.png|thumb|Spectrograms for HKHS3]]
|-
| [[File:2014-svs-iiy1.png|thumb|Spectrograms for IIY1]]
| [[File:2014-svs-iiy2.png|thumb|Spectrograms for IIY2]]
| [[File:2014-svs-jl1.png|thumb|Spectrograms for JL1]]
| [[File:2014-svs-lfr1.png|thumb|Spectrograms for LFR1]]
|-
| [[File:2014-svs-rna1.png|thumb|Spectrograms for RNA1]]
| [[File:2014-svs-rp1.png|thumb|Spectrograms for RP1]]
| [[File:2014-svs-yc1.png|thumb|Spectrograms for YC1]]
|}

=== Labels ===

'''a''' = input mixture ''x'' 
'''b''' = ground truth voice for ''x'' 
'''c''' = extracted voice from ''x'' 
'''d''' = input mixture ''y'' 
'''e''' = ground truth voice for ''y'' 
'''f''' = extracted voice from ''y'' 
'''g''' = input mixture ''z'' 
'''h''' = ground truth voice for ''z'' 
'''i''' = extracted voice from ''z'' 

== Runtime Data ==

<csv>2014/svs/runtime.csv</csv>

2015:Singing Voice Separation Results

2016-08-03T08:42:14Z

Tak-Shing Chan: /* Description */

== Introduction ==

=== Description ===

These are the results for the 2015 running of the Singing Voice Separation task set. The evaluation set is kindly provided by [http://mac.citi.sinica.edu.tw/ikala/ iKala]. If you need to cite this page, please also cite T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722. For more information about this task set please refer to the [[2015:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! FJ1
| MIREX 2015 Submission for Singing Voice Separation || style="text-align: center;" | - || Zhe-Cheng Fan, Jyh-Shing Roger Jang
|-
! FJ2
| MIREX 2015 Submission for Singing Voice Separation || style="text-align: center;" | - || Zhe-Cheng Fan, Jyh-Shing Roger Jang
|-
! IIY3
| MIREX2015: Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/IIY3.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! IIY4
| MIREX2015: Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/IIY4.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! MD3
| An Ensemble Method for Learning to Extract Vocals from Polyphonic Musical Audio || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/MD3.pdf PDF] || Matt McVicar, Tijl De Bie
|-
! MD4
| An Ensemble Method for Learning to Extract Vocals from Polyphonic Musical Audio || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/MD4.pdf PDF] || Matt McVicar, Tijl De Bie
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (m)
|-
| FJ1 || 6.8236 || 10.135 || 3.4014
|-
| FJ2 || 6.3487 || 9.8678 || 2.9135
|-
| IIY3 || 4.9862 || 8.2138 || 99.1737
|-
| IIY4 || 5.3953 || 8.77 || 46.6621
|-
| MD3 || 2.9831 || 6.3671 || 121.6655
|-
| MD4 || 3.1022 || 7.4657 || 121.1348
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2015/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2015/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2015-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2015/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2015/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2015-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2015/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2015/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2015-svs-sar.png]]

== Runtime Data ==

<csv>2015/svs/runtime.csv</csv>

2016:Singing Voice Separation Results

2016-08-03T08:41:29Z

Tak-Shing Chan: /* Description */

== Introduction ==

=== Description ===

These are the results for the 2016 running of the Singing Voice Separation task set. The evaluation set is kindly provided by [http://mac.citi.sinica.edu.tw/ikala/ iKala]. If you need to cite this page, please also cite T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722. For more information about this task set please refer to the [[2016:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! GD1
| Harmonic Modeling of Singing Voice for Source Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/GD1.pdf PDF] || Georgi Dzhambazov
|-
! HC1
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/HC1.pdf PDF] || Yi-Chun Huang, Tai-Shih Chi
|-
! LCP1
| Deep Clustering for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/LCP1.pdf PDF] || Yi Luo, Zhuo Chen, John R. Hershey, Jonathan Le Roux, Daniel P. W. Ellis
|-
! LCP2
| Deep Clustering for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/LCP2.pdf PDF] || Yi Luo, Zhuo Chen, John R. Hershey, Jonathan Le Roux, Daniel P. W. Ellis
|-
! MC2
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | - || Marius Miron, Pritish Chandna
|-
! MC3
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | - || Marius Miron, Pritish Chandna
|-
! RSGP1
| Singing Voice Separation Using Deep Neural Networks and F0 Estimation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/RSGP1.pdf PDF] || Gerard Roma, Emad M. Grais, Andrew J. R. Simpson, Mark D. Plumbley
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (m)
|-
| GD1 || -2.2810 || 0.3954 || 26.4413
|-
| HC1 || 4.6309 || 7.8180 || 28.9727
|-
| LCP1 || 6.0726 || 10.9256 || 37.8235
|-
| LCP2 || 6.3414 || 11.1878 || 32.4800
|-
| MC2 || 5.2891 || 9.6678 || 34.8084
|-
| MC3 || 5.4920 || 9.8049 || 36.7194
|-
| RSGP1 || 3.2589 || 8.7664 || 32.3578
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-sar.png]]

== Runtime Data ==

<csv>2016/svs/runtime.csv</csv>

2016:Singing Voice Separation Results

2016-08-01T08:23:42Z

Tak-Shing Chan:

== Introduction ==

=== Description ===

These are the results for the 2016 running of the Singing Voice Separation task set. For more information about this task set please refer to the [[2016:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! GD1
| Harmonic Modeling of Singing Voice for Source Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/GD1.pdf PDF] || Georgi Dzhambazov
|-
! HC1
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/HC1.pdf PDF] || Yi-Chun Huang, Tai-Shih Chi
|-
! LCP1
| Deep Clustering for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/LCP1.pdf PDF] || Yi Luo, Zhuo Chen, John R. Hershey, Jonathan Le Roux, Daniel P. W. Ellis
|-
! LCP2
| Deep Clustering for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/LCP2.pdf PDF] || Yi Luo, Zhuo Chen, John R. Hershey, Jonathan Le Roux, Daniel P. W. Ellis
|-
! MC2
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | - || Marius Miron, Pritish Chandna
|-
! MC3
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | - || Marius Miron, Pritish Chandna
|-
! RSGP1
| Singing Voice Separation Using Deep Neural Networks and F0 Estimation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/RSGP1.pdf PDF] || Gerard Roma, Emad M. Grais, Andrew J. R. Simpson, Mark D. Plumbley
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (m)
|-
| GD1 || -2.2810 || 0.3954 || 26.4413
|-
| HC1 || 4.6309 || 7.8180 || 28.9727
|-
| LCP1 || 6.0726 || 10.9256 || 37.8235
|-
| LCP2 || 6.3414 || 11.1878 || 32.4800
|-
| MC2 || 5.2891 || 9.6678 || 34.8084
|-
| MC3 || 5.4920 || 9.8049 || 36.7194
|-
| RSGP1 || 3.2589 || 8.7664 || 32.3578
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-sar.png]]

== Runtime Data ==

<csv>2016/svs/runtime.csv</csv>

2016:Singing Voice Separation Results

2016-08-01T08:20:49Z

Tak-Shing Chan:

== Introduction ==

=== Description ===

These are the results for the 2016 running of the Singing Voice Separation task set. For more information about this task set please refer to the [[2016:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! GD1
| Harmonic Modeling of Singing Voice for Source Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/GD1.pdf PDF] || Georgi Dzhambazov
|-
! HC1
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/HC1.pdf PDF] || Yi-Chun Huang, Tai-Shih Chi
|-
! LCP1
| Deep Clustering for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/LCP1.pdf PDF] || Yi Luo, Zhuo Chen, John R. Hershey, Jonathan Le Roux, Daniel P. W. Ellis
|-
! LCP2
| Deep Clustering for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/LCP2.pdf PDF] || Yi Luo, Zhuo Chen, John R. Hershey, Jonathan Le Roux, Daniel P. W. Ellis
|-
! MC2
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | - || Marius Miron, Pritish Chandna
|-
! MC3
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | - || Marius Miron, Pritish Chandna
|-
! RSGP1
| Singing Voice Separation using Deep Neural Networks and F0 Estimation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/RSGP1.pdf PDF] || Gerard Roma, Emad M. Grais, Andrew J. R. Simpson, Mark D. Plumbley
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (m)
|-
| GD1 || -2.2810 || 0.3954 || 26.4413
|-
| HC1 || 4.6309 || 7.8180 || 28.9727
|-
| LCP1 || 6.0726 || 10.9256 || 37.8235
|-
| LCP2 || 6.3414 || 11.1878 || 32.4800
|-
| MC2 || 5.2891 || 9.6678 || 34.8084
|-
| MC3 || 5.4920 || 9.8049 || 36.7194
|-
| RSGP1 || 3.2589 || 8.7664 || 32.3578
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-sar.png]]

== Runtime Data ==

<csv>2016/svs/runtime.csv</csv>

File:2016-svs-sar.png

2016-08-01T08:04:16Z

Tak-Shing Chan: SAR

SAR

File:2016-svs-sir.png

2016-08-01T08:03:51Z

Tak-Shing Chan: SIR

SIR

File:2016-svs-nsdr.png

2016-08-01T08:03:22Z

Tak-Shing Chan: NSDR

NSDR

2016:Singing Voice Separation Results

2016-08-01T08:01:50Z

Tak-Shing Chan: Created page with "== Introduction == === Description === These are the results for the 2016 running of the Singing Voice Separation task set. For more information about this task set please refe..."

== Introduction ==

=== Description ===

These are the results for the 2016 running of the Singing Voice Separation task set. For more information about this task set please refer to the [[2016:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! GD1
| HARMONIC MODELING OF SINGING VOICE FOR SOURCE SEPARATION || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/GD1.pdf PDF] || Georgi Dzhambazov
|-
! HC1
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/HC1.pdf PDF] || Yi-Chun Huang, Tai-Shih Chi
|-
! LCP1
| DEEP CLUSTERING FOR SINGING VOICE SEPARATION || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/LCP1.pdf PDF] || Yi Luo, Zhuo Chen, John R. Hershey, Jonathan Le Roux, Daniel P. W. Ellis
|-
! LCP2
| DEEP CLUSTERING FOR SINGING VOICE SEPARATION || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/LCP2.pdf PDF] || Yi Luo, Zhuo Chen, John R. Hershey, Jonathan Le Roux, Daniel P. W. Ellis
|-
! MC2
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | - || Marius Miron, Pritish Chandna
|-
! MC3
| MIREX 2016 Submission for Singing Voice Separation || style="text-align: center;" | - || Marius Miron, Pritish Chandna
|-
! RSGP1
| MIREX || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2016/RSGP1.pdf PDF] || Gerard Roma, Emad M. Grais, Andrew J. R. Simpson, Mark D. Plumbley
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (m)
|-
| GD1 || -2.2810 || 0.3954 || 26.4413
|-
| HC1 || 4.6309 || 7.8180 || 28.9727
|-
| LCP1 || 6.0726 || 10.9256 || 37.8235
|-
| LCP2 || 6.3414 || 11.1878 || 32.4800
|-
| MC2 || 5.2891 || 9.6678 || 34.8084
|-
| MC3 || 5.4920 || 9.8049 || 36.7194
|-
| RSGP1 || 3.2589 || 8.7664 || 32.3578
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2016/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2016/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2016-svs-sar.png]]

== Runtime Data ==

<csv>2016/svs/runtime.csv</csv>

2016:Singing Voice Separation

2016-04-29T07:54:45Z

Tak-Shing Chan: /* Data */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms (these are the hidden parts of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset). If your algorithm is a supervised one, you are welcome to use the public part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset for training. In addition, you can train with 30-second segments from the SiSEC [https://sisec.inria.fr/home/2016-professionally-produced-music-recordings/ MUS] challenge.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

For more information about the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset, see T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722.

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0] (but with the permutation part removed, as the ability to classify signals is part of the singing voice separation challenge). Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2015 Submission Instructions below). For supervised submissions, please provide training details in the extended abstract (e.g. datasets used).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2016 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2015:Singing Voice Separation

2016-04-29T07:52:53Z

Tak-Shing Chan: /* Data */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms (these are the hidden parts of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset). If your algorithm is a supervised one, you are welcome to use the public part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset for training. In addition, you can train with 30-second segments from the SiSEC [https://sisec.inria.fr/sisec-2015/2015-professionally-produced-music-recordings/ MUS] challenge.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

For more information about the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset, see T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722.

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0] (but with the permutation part removed, as the ability to classify signals is part of the singing voice separation challenge). Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2015 Submission Instructions below). For supervised submissions, please provide training details in the extended abstract (e.g. datasets used).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2015 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2016:Singing Voice Separation

2016-04-29T07:44:11Z

Tak-Shing Chan: /* Data */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms (these are the hidden parts of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset). If your algorithm is a supervised one, you are welcome to use the public part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset for training. In addition, you can train with 30-second segments from the SiSEC [https://sisec.inria.fr/professionally-produced-music-recordings/ MUS] challenge.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

For more information about the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset, see T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722.

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0] (but with the permutation part removed, as the ability to classify signals is part of the singing voice separation challenge). Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2015 Submission Instructions below). For supervised submissions, please provide training details in the extended abstract (e.g. datasets used).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2016 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2015:Singing Voice Separation

2016-04-29T07:43:32Z

Tak-Shing Chan: /* Data */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms (these are the hidden parts of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset). If your algorithm is a supervised one, you are welcome to use the public part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset for training. In addition, you can train with 30-second segments from the SiSEC [https://sisec.inria.fr/professionally-produced-music-recordings/ MUS] challenge.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

For more information about the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset, see T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722.

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0] (but with the permutation part removed, as the ability to classify signals is part of the singing voice separation challenge). Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2015 Submission Instructions below). For supervised submissions, please provide training details in the extended abstract (e.g. datasets used).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2015 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2014:Singing Voice Separation

2016-04-29T07:42:04Z

Tak-Shing Chan: /* Data */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms (these are the hidden parts of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset). If your algorithm is a supervised one, you are welcome to use the public part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset for training.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

For more information about the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset, see T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang, "Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722.

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0]. Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2014 Submission Instructions below).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2014 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2014:Singing Voice Separation

2016-04-29T07:37:04Z

Tak-Shing Chan: /* Data */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

This collection comprises of the hidden part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala dataset]. See T.-S. Chan, T.-C. Yeh, Z.-C. Fan, H.-W. Chen, L. Su, Y.-H. Yang, and R. Jang,
"Vocal activity informed singing voice separation with the iKala dataset," in Proc. IEEE Int. Conf. Acoust., Speech and Signal Process., 2015, pp. 718-722.

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0]. Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2014 Submission Instructions below).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2014 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2014:Singing Voice Separation Results

2015-11-19T02:27:37Z

Tak-Shing Chan:

== Introduction ==

=== Description ===

These are the results for the 2014 running of the Singing Voice Separation task set. For more information about this task set please refer to the [[2014:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! GW1
| Bayesian Singing-Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/GW1.pdf PDF] || Guan-Xiang Wang, Po-Kai Yang, Chung-Chien Hsu, Jen-Tzung Chien
|-
! HKHS1
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS1.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! HKHS2
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS2.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! HKHS3
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS3.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! IIY1
| Singing Voice Separation and Vocal F0 Estimation based on Robust PCA and Subharmonic Summation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/IIY1.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! IIY2
| Singing Voice Separation and Vocal F0 Estimation based on Robust PCA and Subharmonic Summation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/IIY2.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! JL1
| Singing Voice Separation Based on Sparse Nature and Spectral/Temporal Discontinuity || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/JL1.pdf PDF] || Il-Young Jeong, Kyogu Lee
|-
! LFR1
| Kernel Additive Modelling with light models || style="text-align: center;" | [http://dx.doi.org/10.1109/ICASSP.2015.7177935 PDF] || Antoine Liutkus, Derry Fitzgerald, Zafar Rafii
|-
! RNA1
| Singing Voice Separation using Adaptive Window Harmonic Sinusoidal Modeling || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/RNA1.pdf PDF] || Preeti Rao, Nagesh Nayak, Sharath Adavanne
|-
! RP1
| REPET-SIM for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/RP1.pdf PDF] || Zafar Rafii, Bryan Pardo
|-
! YC1
| MIREX 2014 Submission for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/YC1.pdf PDF] || Frederick Yen, Tai-Shih Chi
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (hh)
|-
| GW1 || 2.8861 || 5.2549 || 24
|-
| HKHS1 || -1.3988 || 0.3483 || 06
|-
| HKHS2 || -1.9413 || 0.5239 || 06
|-
| HKHS3 || -2.4807 || 0.1414 || 06
|-
| IIY1 || 4.2190 || 7.7893 || 02
|-
| IIY2 || 4.4764 || 7.8661 || 02
|-
| JL1 || 4.1564 || 5.6304 || 01
|-
| LFR1 || 0.6499 || 3.0867 || 03
|-
| RNA1 || 3.6915 || 7.3153 || 06
|-
| RP1 || 2.8602 || 5.0306 || 01
|-
| YC1 || -0.8202 || -3.1150 || 13
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-sar.png]]

== Individual Spectrograms ==

As the MIREX test set is private, we use three other songs with similar characteristics to demonstrate the algorithms.

{| border="1" cellspacing="0" style="text-align: left;"
|-
| [[File:2014-svs-gw1.png|thumb|Spectrograms for GW1]]
| [[File:2014-svs-hkhs1.png|thumb|Spectrograms for HKHS1]]
| [[File:2014-svs-hkhs2.png|thumb|Spectrograms for HKHS2]]
| [[File:2014-svs-hkhs3.png|thumb|Spectrograms for HKHS3]]
|-
| [[File:2014-svs-iiy1.png|thumb|Spectrograms for IIY1]]
| [[File:2014-svs-iiy2.png|thumb|Spectrograms for IIY2]]
| [[File:2014-svs-jl1.png|thumb|Spectrograms for JL1]]
| [[File:2014-svs-lfr1.png|thumb|Spectrograms for LFR1]]
|-
| [[File:2014-svs-rna1.png|thumb|Spectrograms for RNA1]]
| [[File:2014-svs-rp1.png|thumb|Spectrograms for RP1]]
| [[File:2014-svs-yc1.png|thumb|Spectrograms for YC1]]
|}

=== Labels ===

'''a''' = input mixture ''x'' 
'''b''' = ground truth voice for ''x'' 
'''c''' = extracted voice from ''x'' 
'''d''' = input mixture ''y'' 
'''e''' = ground truth voice for ''y'' 
'''f''' = extracted voice from ''y'' 
'''g''' = input mixture ''z'' 
'''h''' = ground truth voice for ''z'' 
'''i''' = extracted voice from ''z'' 

== Runtime Data ==

<csv>2014/svs/runtime.csv</csv>

2015:Singing Voice Separation Results

2015-10-17T06:00:54Z

Tak-Shing Chan: /* Legend */

== Introduction ==

=== Description ===

These are the results for the 2015 running of the Singing Voice Separation task set. For more information about this task set please refer to the [[2015:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! FJ1
| MIREX 2015 Submission for Singing Voice Separation || style="text-align: center;" | - || Zhe-Cheng Fan, Jyh-Shing Roger Jang
|-
! FJ2
| MIREX 2015 Submission for Singing Voice Separation || style="text-align: center;" | - || Zhe-Cheng Fan, Jyh-Shing Roger Jang
|-
! IIY3
| MIREX2015: Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/IIY3.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! IIY4
| MIREX2015: Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/IIY4.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! MD3
| An Ensemble Method for Learning to Extract Vocals from Polyphonic Musical Audio || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/MD3.pdf PDF] || Matt McVicar, Tijl De Bie
|-
! MD4
| An Ensemble Method for Learning to Extract Vocals from Polyphonic Musical Audio || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/MD4.pdf PDF] || Matt McVicar, Tijl De Bie
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (m)
|-
| FJ1 || 6.8236 || 10.135 || 3.4014
|-
| FJ2 || 6.3487 || 9.8678 || 2.9135
|-
| IIY3 || 4.9862 || 8.2138 || 99.1737
|-
| IIY4 || 5.3953 || 8.77 || 46.6621
|-
| MD3 || 2.9831 || 6.3671 || 121.6655
|-
| MD4 || 3.1022 || 7.4657 || 121.1348
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2015/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2015/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2015-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2015/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2015/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2015-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2015/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2015/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2015-svs-sar.png]]

== Runtime Data ==

<csv>2015/svs/runtime.csv</csv>

2015:Singing Voice Separation Results

2015-10-06T04:14:58Z

Tak-Shing Chan:

== Introduction ==

=== Description ===

These are the results for the 2015 running of the Singing Voice Separation task set. For more information about this task set please refer to the [[2015:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! FJ1
| MIREX 2015 Submission for Singing Voice Separation || style="text-align: center;" | - || Zhe-Cheng Fan, Jyh-Shing Roger Jang
|-
! FJ2
| MIREX 2015 Submission for Singing Voice Separation || style="text-align: center;" | - || Zhe-Cheng Fan, Jyh-Shing Roger Jang
|-
! IIY3
| MIREX 2015 Submission for Singing Voice Separation || style="text-align: center;" | - || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! IIY4
| MIREX 2015 Submission for Singing Voice Separation || style="text-align: center;" | - || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! MD3
| An Ensemble Method for Learning to Extract Vocals from Polyphonic Musical Audio || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/MD3.pdf PDF] || Matt McVicar, Tijl De Bie
|-
! MD4
| An Ensemble Method for Learning to Extract Vocals from Polyphonic Musical Audio || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2015/MD4.pdf PDF] || Matt McVicar, Tijl De Bie
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (m)
|-
| FJ1 || 6.8236 || 10.135 || 3.4014
|-
| FJ2 || 6.3487 || 9.8678 || 2.9135
|-
| IIY3 || 4.9862 || 8.2138 || 99.1737
|-
| IIY4 || 5.3953 || 8.77 || 46.6621
|-
| MD3 || 2.9831 || 6.3671 || 121.6655
|-
| MD4 || 3.1022 || 7.4657 || 121.1348
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2015/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2015/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2015-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2015/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2015/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2015-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2015/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2015/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2015-svs-sar.png]]

== Runtime Data ==

<csv>2015/svs/runtime.csv</csv>

File:2015-svs-sir.png

2015-10-06T03:47:43Z

Tak-Shing Chan: SIR

SIR

File:2015-svs-sar.png

2015-10-06T03:47:28Z

Tak-Shing Chan: SAR

SAR

File:2015-svs-nsdr.png

2015-10-06T03:47:08Z

Tak-Shing Chan: NSDR

NSDR

2015:Singing Voice Separation Results

2015-10-05T15:55:55Z

Tak-Shing Chan: Created page with "== Introduction == === Description === These are the results for the 2015 running of the Singing Voice Separation task set. For more information about this task set please refe..."

== Introduction ==

=== Description ===

These are the results for the 2015 running of the Singing Voice Separation task set. For more information about this task set please refer to the [[2015:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! FJ1
| Submission name || style="text-align: center;" | - || Contributors
|-
! FJ2
| Submission name || style="text-align: center;" | - || Contributors
|-
! IIY3
| Submission name || style="text-align: center;" | - || Contributors
|-
! IIY4
| Submission name || style="text-align: center;" | - || Contributors
|-
! MD3
| Submission name || style="text-align: center;" | - || Contributors
|-
! MD4
| Submission name || style="text-align: center;" | - || Contributors
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (m)
|-
| FJ1 || || ||
|-
| FJ2 || || ||
|-
| IIY3 || || ||
|-
| IIY4 || || ||
|-
| MD3 || || ||
|-
| MD4 || || ||
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2015/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2015/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2015-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2015/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2015/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2015-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2015/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2015/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2015-svs-sar.png]]

== Runtime Data ==

<csv>2015/svs/runtime.csv</csv>

2015:Singing Voice Separation

2015-08-10T08:25:36Z

Tak-Shing Chan: /* Data */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms (these are the hidden parts of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset). If your algorithm is a supervised one, you are welcome to use the public part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset for training. In addition, you can train with 30-second segments from the SiSEC [https://sisec.inria.fr/professionally-produced-music-recordings/ MUS] challenge.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0] (but with the permutation part removed, as the ability to classify signals is part of the singing voice separation challenge). Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2015 Submission Instructions below). For supervised submissions, please provide training details in the extended abstract (e.g. datasets used).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2015 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2015:Singing Voice Separation

2015-08-10T08:24:03Z

Tak-Shing Chan: /* Evaluation */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms. These are the hidden parts of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset. If your algorithm is a supervised one, you are welcome to use the public part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset. Alternatively, or in addition, you can train with 30-second segments from the SiSEC [https://sisec.inria.fr/professionally-produced-music-recordings/ MUS] challenge.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0] (but with the permutation part removed, as the ability to classify signals is part of the singing voice separation challenge). Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2015 Submission Instructions below). For supervised submissions, please provide training details in the extended abstract (e.g. datasets used).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2015 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2015:Singing Voice Separation

2015-08-10T08:23:48Z

Tak-Shing Chan: /* Data */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms. These are the hidden parts of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset. If your algorithm is a supervised one, you are welcome to use the public part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset. Alternatively, or in addition, you can train with 30-second segments from the SiSEC [https://sisec.inria.fr/professionally-produced-music-recordings/ MUS] challenge.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0] (NEW: but with the perm part removed, as the ability to classify signals is part of the singing voice separation challenge). Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2015 Submission Instructions below). For supervised submissions, please provide training details in the extended abstract (e.g. datasets used).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2015 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2015:Singing Voice Separation

2015-08-10T08:17:22Z

Tak-Shing Chan: /* Submission format */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms. These are the hidden parts of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset. For training purposes, you are welcome to use the public part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset. Alternatively (or in addition), you can train with 30-second segments from the SiSEC [https://sisec.inria.fr/professionally-produced-music-recordings/ MUS] challenge.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0] (NEW: but with the perm part removed, as the ability to classify signals is part of the singing voice separation challenge). Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2015 Submission Instructions below). For supervised submissions, please provide training details in the extended abstract (e.g. datasets used).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2015 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2015:Singing Voice Separation

2015-08-10T08:10:46Z

Tak-Shing Chan: /* Data */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms. These are the hidden parts of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset. For training purposes, you are welcome to use the public part of the [http://mac.citi.sinica.edu.tw/ikala/ iKala] dataset. Alternatively (or in addition), you can train with 30-second segments from the SiSEC [https://sisec.inria.fr/professionally-produced-music-recordings/ MUS] challenge.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0] (NEW: but with the perm part removed, as the ability to classify signals is part of the singing voice separation challenge). Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2015 Submission Instructions below).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2015 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

2015:Singing Voice Separation

2015-08-10T08:02:51Z

Tak-Shing Chan: /* Evaluation */

== Description ==

The singing voice separation task solicits competing entries to blindly separate the singer's voice from pop music recordings. The entries are evaluated using standard metrics (see Evaluation below).

=== Task specific mailing list ===

All discussions take place on the MIREX "EvalFest" list. If you have an question or comment, simply include the task name in the subject heading.

== Data ==

A collection of 100 clips of recorded pop music (vocals plus music) are used to evaluate the singing voice separation algorithms.

Collection statistics:

# Size of collection: 100 clips
# Audio details: 16-bit, mono, 44.1kHz, WAV
# Duration of each clip: 30 seconds

== Evaluation ==

For evaluation we use [http://hal.inria.fr/inria-00630985/PDF/vincent_SigPro11.pdf Vincent ''et al.'''s (2012)] Source to Distortion Ratio (SDR), Source to Interferences Ratio (SIR), and Sources to Artifacts Ratio (SAR), as implemented by [http://bass-db.gforge.inria.fr/bss_eval/bss_eval_sources.m bss_eval_sources.m] in [http://bass-db.gforge.inria.fr/bss_eval/ BSS Eval Version 3.0] (NEW: but with the perm part removed, as the ability to classify signals is part of the singing voice separation challenge). Everything will be normalized to enable a fairer evaluation. More specifically, their function will be invoked as follows:

>> trueVoice = wavread('trueVoice.wav');
>> trueKaraoke = wavread('trueKaraoke.wav');
>> trueMixed = trueVoice + trueKaraoke;
>> [estimatedVoice, estimatedKaraoke] = wrapper_function_calling_your_separation_algorithm(trueMixed);
>> [SDR, SIR, SAR] = bss_eval_sources([estimatedVoice estimatedKaraoke]' / norm(estimatedVoice + estimatedKaraoke), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> [NSDR, NSIR, NSAR] = bss_eval_sources([trueMixed trueMixed]' / norm(trueMixed + trueMixed), [trueVoice trueKaraoke]' / norm(trueVoice + trueKaraoke));
>> NSDR = SDR - NSDR;
>> NSIR = SIR - NSIR;
>> NSAR = SAR - NSAR;

The final scores will be determined by the mean over all 100 clips (note that GSIR and GSAR are not normalized):

<math>GNSDR=\frac{\sum_{i=1}^{100}NSDR_i}{100}</math>,

<math>GSIR=\frac{\sum_{i=1}^{100}SIR_i}{100}</math>,

<math>GSAR=\frac{\sum_{i=1}^{100}SAR_i}{100}</math>.

In addition, sd, min, max and median will also be reported.

== Submission format ==

Participants are required to submit an entry that takes in an input filename (full native pathname ending in *.wav) and an output directory as arguments. The entries must send their voice-separated outputs to *-voice.wav and *-music.wav under the output directory. For example:

function singing_voice_separation(infile, outdir)
[~, name, ext] = fileparts(infile);
your_algorithm(infile, fullfile(outdir, [name '-voice' ext]), fullfile(outdir, [name '-music' ext]));

function your_algorithm(infile, voiceoutfile, musicoutfile)
mixed = wavread(infile);

% Insert your algorithm here

wavwrite(voice, 44100, voiceoutfile);
wavwrite(music, 44100, musicoutfile);

If scratch space is required, please use the three-argument format instead:

function singing_voice_separation(infile, outdir, tmpdir)

Following the convention of other MIREX tasks, an extended abstract is also required (see MIREX 2015 Submission Instructions below).

== Packaging submissions ==
All submissions should be statically linked to all libraries (the presence of dynamically linked libraries cannot be guaranteed).
# Be sure to follow the [[2006:Best Coding Practices for MIREX | Best Coding Practices for MIREX]].
# Be sure to follow the [[MIREX 2015 Submission Instructions]]. For example, under '''Very Important Things to Note''', Clause 6 states that if you plan to submit more than one algorithm or algorithm variant to a given task, EACH algorithm or variant needs its own complete submission to be made including the README and binary bundle upload. Each package will be given its own unique identifier. Tell us in the README the priority of a given algorithm in case we have to limit a task to only one or two algorithms/variants per submitter/team. [Note: our current limit is two entries per team.]

All submissions should include a README file including the following the information:
# Command line calling format for all executables and an example formatted set of commands
# Number of threads/cores used or whether this should be specified on the command line
# Expected memory footprint
# Expected runtime
# Approximately how much scratch disk space will the submission need to store any feature/cache files?
# Any required environments/architectures (and versions), e.g. python, java, bash, matlab.
# Any special notice regarding to running your algorithm

Note that the information that you place in the README file is '''extremely''' important in ensuring that your submission is evaluated properly.

== Time and hardware limits ==

Due to the potentially high number of particpants in this and other audio tasks, hard limits on the runtime of submissions are specified.

A hard limit of 24 hours will be imposed on runs. Submissions that exceed this runtime may not receive a result.

== Potential Participants ==

name / email

User:Tak-Shing Chan

2015-05-06T06:13:39Z

Tak-Shing Chan:

Tak-Shing T. Chan received the Ph.D. degree from the University of London, London, UK in 2008. From 2006 to 2008, he was a Scientific Programmer at the University of Sheffield. In 2011, he worked as a Research Associate at the Hong Kong Polytechnic University. He is currently a Postdoctoral Fellow at the Academia Sinica, Taipei, Taiwan.

2015:Task Captains

2015-03-27T02:13:34Z

Tak-Shing Chan:

Like ISMIR 2014, we are prepared to improve the distribution of tasks for the upcoming MIREX 2015. To do so, we really need leaders to help us organize and run each task.

To volunteer to lead one or more tasks, please add your name in the "Captains" column.

What does it mean to lead a task?
* Update wiki pages as needed
* Communicate with submitters and troubleshooting submissions
* Execution and evaluation of submissions
* Publishing final results

Due to the proprietary nature of much of the data, the submission system, evaluation framework, and most of the datasets will continue to be hosted by IMIRSEL. However, we are prepared to provide access to task organizers to manage and run submissions on the IMIRSEL systems.

{| class="wikitable" style="margin-left: 20px"
!ID !! Task !! Captain(s)
|-
|abt
|[[2015:Audio Beat Tracking]]
|
|-
|ace
|[[2015:Audio Chord Estimation]]
|
|-
|act
|[[2015:Audio Classification (Train/Test) Tasks]]
|
|-
|acs
|[[2015:Audio Cover Song Identification]]
|
|-
|ade
|[[2015:Audio Downbeat Estimation]]
|
|-
|akd
|[[2015:Audio Key Detection]]
|
|-
|ame
|[[2015:Audio Melody Extraction]]
|
|-
|ams
|[[2015:Audio Music Similarity and Retrieval]]
|
|-
|aod
|[[2015:Audio Onset Detection]]
|
|-
|ate
|[[2015:Audio Tempo Estimation]]
|
|-
|atg
|[[2015:Audio Tag Classification]]
|
|-
|mf0
|[[2015:Multiple Fundamental Frequency Estimation & Tracking]]
|
|-
|qbsh
|[[2015:Query by Singing/Humming]]
|
|-
|qbt
|[[2015:Query by Tapping]]
|
|-
|scofo
|[[2015:Real-time Audio to Score Alignment (a.k.a Score Following)]]
|
|-
|sms
|[[2015:Symbolic Melodic Similarity]]
|
|-
|struct
|[[2015:Structural Segmentation]]
|
|-
|drts
|[[2015:Discovery of Repeated Themes & Sections]]
|
|-
|afp
|[[2015:Audio_Fingerprinting]]
|
|-
|svs
|[[2015:Singing_Voice_Separation]]
|Tak-Shing Chan, Li Su, Yi-Hsuan Yang
|-
|kgc
|[[2015:Audio K-POP Genre Classification]]
|
|-
|kmc
|[[2015:Audio K-POP Mood Classification]]
|
|}

User:Tak-Shing Chan

2015-02-10T07:42:07Z

Tak-Shing Chan:

Tak-Shing Chan received the Ph.D. degree in computing from the University of London in 2008. From 2006 to 2008, he was a Scientific Programmer at the University of Sheffield. In 2011, he worked as a Research Associate at the Hong Kong Polytechnic University. He is currently a Postdoctoral Fellow at the Academia Sinica. His research interests include sparse coding, signal processing, music cognition, and distributed systems.

2014:Singing Voice Separation Results

2014-10-17T10:19:54Z

Tak-Shing Chan: /* Labels */

== Introduction ==

=== Description ===

These are the results for the 2014 running of the Singing Voice Separation task set. For more information about this task set please refer to the [[2014:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! GW1
| Bayesian Singing-Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/GW1.pdf PDF] || Guan-Xiang Wang, Po-Kai Yang, Chung-Chien Hsu, Jen-Tzung Chien
|-
! HKHS1
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS1.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! HKHS2
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS2.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! HKHS3
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS3.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! IIY1
| Singing Voice Separation and Vocal F0 Estimation based on Robust PCA and Subharmonic Summation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/IIY1.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! IIY2
| Singing Voice Separation and Vocal F0 Estimation based on Robust PCA and Subharmonic Summation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/IIY2.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! JL1
| Singing Voice Separation Based on Sparse Nature and Spectral/Temporal Discontinuity || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/JL1.pdf PDF] || Il-Young Jeong, Kyogu Lee
|-
! LFR1
| Kernel Additive Modelling with light models || style="text-align: center;" | - || Antoine Liutkus, Derry Fitzgerald, Zafar Rafii
|-
! RNA1
| Singing Voice Separation using Adaptive Window Harmonic Sinusoidal Modeling || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/RNA1.pdf PDF] || Preeti Rao, Nagesh Nayak, Sharath Adavanne
|-
! RP1
| REPET-SIM for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/RP1.pdf PDF] || Zafar Rafii, Bryan Pardo
|-
! YC1
| MIREX 2014 Submission for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/YC1.pdf PDF] || Frederick Yen, Tai-Shih Chi
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (hh)
|-
| GW1 || 2.8861 || 5.2549 || 24
|-
| HKHS1 || -1.3988 || 0.3483 || 06
|-
| HKHS2 || -1.9413 || 0.5239 || 06
|-
| HKHS3 || -2.4807 || 0.1414 || 06
|-
| IIY1 || 4.2190 || 7.7893 || 02
|-
| IIY2 || 4.4764 || 7.8661 || 02
|-
| JL1 || 4.1564 || 5.6304 || 01
|-
| LFR1 || 0.6499 || 3.0867 || 03
|-
| RNA1 || 3.6915 || 7.3153 || 06
|-
| RP1 || 2.8602 || 5.0306 || 01
|-
| YC1 || -0.8202 || -3.1150 || 13
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-sar.png]]

== Individual Spectrograms ==

As the MIREX test set is private, we use three other songs with similar characteristics to demonstrate the algorithms.

{| border="1" cellspacing="0" style="text-align: left;"
|-
| [[File:2014-svs-gw1.png|thumb|Spectrograms for GW1]]
| [[File:2014-svs-hkhs1.png|thumb|Spectrograms for HKHS1]]
| [[File:2014-svs-hkhs2.png|thumb|Spectrograms for HKHS2]]
| [[File:2014-svs-hkhs3.png|thumb|Spectrograms for HKHS3]]
|-
| [[File:2014-svs-iiy1.png|thumb|Spectrograms for IIY1]]
| [[File:2014-svs-iiy2.png|thumb|Spectrograms for IIY2]]
| [[File:2014-svs-jl1.png|thumb|Spectrograms for JL1]]
| [[File:2014-svs-lfr1.png|thumb|Spectrograms for LFR1]]
|-
| [[File:2014-svs-rna1.png|thumb|Spectrograms for RNA1]]
| [[File:2014-svs-rp1.png|thumb|Spectrograms for RP1]]
| [[File:2014-svs-yc1.png|thumb|Spectrograms for YC1]]
|}

=== Labels ===

'''a''' = input mixture ''x'' 
'''b''' = ground truth voice for ''x'' 
'''c''' = extracted voice from ''x'' 
'''d''' = input mixture ''y'' 
'''e''' = ground truth voice for ''y'' 
'''f''' = extracted voice from ''y'' 
'''g''' = input mixture ''z'' 
'''h''' = ground truth voice for ''z'' 
'''i''' = extracted voice from ''z'' 

== Runtime Data ==

<csv>2014/svs/runtime.csv</csv>

2014:Singing Voice Separation Results

2014-10-17T09:57:43Z

Tak-Shing Chan: /* Legend */

== Introduction ==

=== Description ===

These are the results for the 2014 running of the Singing Voice Separation task set. For more information about this task set please refer to the [[2014:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! GW1
| Bayesian Singing-Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/GW1.pdf PDF] || Guan-Xiang Wang, Po-Kai Yang, Chung-Chien Hsu, Jen-Tzung Chien
|-
! HKHS1
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS1.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! HKHS2
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS2.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! HKHS3
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS3.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! IIY1
| Singing Voice Separation and Vocal F0 Estimation based on Robust PCA and Subharmonic Summation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/IIY1.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! IIY2
| Singing Voice Separation and Vocal F0 Estimation based on Robust PCA and Subharmonic Summation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/IIY2.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! JL1
| Singing Voice Separation Based on Sparse Nature and Spectral/Temporal Discontinuity || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/JL1.pdf PDF] || Il-Young Jeong, Kyogu Lee
|-
! LFR1
| Kernel Additive Modelling with light models || style="text-align: center;" | - || Antoine Liutkus, Derry Fitzgerald, Zafar Rafii
|-
! RNA1
| Singing Voice Separation using Adaptive Window Harmonic Sinusoidal Modeling || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/RNA1.pdf PDF] || Preeti Rao, Nagesh Nayak, Sharath Adavanne
|-
! RP1
| REPET-SIM for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/RP1.pdf PDF] || Zafar Rafii, Bryan Pardo
|-
! YC1
| MIREX 2014 Submission for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/YC1.pdf PDF] || Frederick Yen, Tai-Shih Chi
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (hh)
|-
| GW1 || 2.8861 || 5.2549 || 24
|-
| HKHS1 || -1.3988 || 0.3483 || 06
|-
| HKHS2 || -1.9413 || 0.5239 || 06
|-
| HKHS3 || -2.4807 || 0.1414 || 06
|-
| IIY1 || 4.2190 || 7.7893 || 02
|-
| IIY2 || 4.4764 || 7.8661 || 02
|-
| JL1 || 4.1564 || 5.6304 || 01
|-
| LFR1 || 0.6499 || 3.0867 || 03
|-
| RNA1 || 3.6915 || 7.3153 || 06
|-
| RP1 || 2.8602 || 5.0306 || 01
|-
| YC1 || -0.8202 || -3.1150 || 13
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-sar.png]]

== Individual Spectrograms ==

As the MIREX test set is private, we use three other songs with similar characteristics to demonstrate the algorithms.

{| border="1" cellspacing="0" style="text-align: left;"
|-
| [[File:2014-svs-gw1.png|thumb|Spectrograms for GW1]]
| [[File:2014-svs-hkhs1.png|thumb|Spectrograms for HKHS1]]
| [[File:2014-svs-hkhs2.png|thumb|Spectrograms for HKHS2]]
| [[File:2014-svs-hkhs3.png|thumb|Spectrograms for HKHS3]]
|-
| [[File:2014-svs-iiy1.png|thumb|Spectrograms for IIY1]]
| [[File:2014-svs-iiy2.png|thumb|Spectrograms for IIY2]]
| [[File:2014-svs-jl1.png|thumb|Spectrograms for JL1]]
| [[File:2014-svs-lfr1.png|thumb|Spectrograms for LFR1]]
|-
| [[File:2014-svs-rna1.png|thumb|Spectrograms for RNA1]]
| [[File:2014-svs-rp1.png|thumb|Spectrograms for RP1]]
| [[File:2014-svs-yc1.png|thumb|Spectrograms for YC1]]
|}

=== Labels ===

'''a''' = input mixture x, '''b''' = ground truth voice for x, '''c''' = extracted voice from x, 
'''d''' = input mixture y, '''e''' = ground truth voice for y, '''f''' = extracted voice from y, 
'''g''' = input mixture z, '''h''' = ground truth voice for z, '''i''' = extracted voice from z 

== Runtime Data ==

<csv>2014/svs/runtime.csv</csv>

2014:Singing Voice Separation Results

2014-10-17T09:56:16Z

Tak-Shing Chan: /* Legend */

== Introduction ==

=== Description ===

These are the results for the 2014 running of the Singing Voice Separation task set. For more information about this task set please refer to the [[2014:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! GW1
| Bayesian Singing-Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/GW1.pdf PDF] || Guan-Xiang Wang, Po-Kai Yang, Chung-Chien Hsu, Jen-Tzung Chien
|-
! HKHS1
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS1.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! HKHS2
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS2.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! HKHS3
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS3.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! IIY1
| Singing Voice Separation and Vocal F0 Estimation based on Robust PCA and Subharmonic Summation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/IYI1.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! IIY2
| Singing Voice Separation and Vocal F0 Estimation based on Robust PCA and Subharmonic Summation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/IYI1.pdf PDF] || Yukara Ikemiya, Katsutoshi Itoyama, Kazuyoshi Yoshii
|-
! JL1
| Singing Voice Separation Based on Sparse Nature and Spectral/Temporal Discontinuity || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/JL1.pdf PDF] || Il-Young Jeong, Kyogu Lee
|-
! LFR1
| Kernel Additive Modelling with light models || style="text-align: center;" | - || Antoine Liutkus, Derry Fitzgerald, Zafar Rafii
|-
! RNA1
| Singing Voice Separation using Adaptive Window Harmonic Sinusoidal Modeling || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/RNA1.pdf PDF] || Preeti Rao, Nagesh Nayak, Sharath Adavanne
|-
! RP1
| REPET-SIM for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/RP1.pdf PDF] || Zafar Rafii, Bryan Pardo
|-
! YC1
| MIREX 2014 Submission for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/YC1.pdf PDF] || Frederick Yen, Tai-Shih Chi
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (hh)
|-
| GW1 || 2.8861 || 5.2549 || 24
|-
| HKHS1 || -1.3988 || 0.3483 || 06
|-
| HKHS2 || -1.9413 || 0.5239 || 06
|-
| HKHS3 || -2.4807 || 0.1414 || 06
|-
| IIY1 || 4.2190 || 7.7893 || 02
|-
| IIY2 || 4.4764 || 7.8661 || 02
|-
| JL1 || 4.1564 || 5.6304 || 01
|-
| LFR1 || 0.6499 || 3.0867 || 03
|-
| RNA1 || 3.6915 || 7.3153 || 06
|-
| RP1 || 2.8602 || 5.0306 || 01
|-
| YC1 || -0.8202 || -3.1150 || 13
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-sar.png]]

== Individual Spectrograms ==

As the MIREX test set is private, we use three other songs with similar characteristics to demonstrate the algorithms.

{| border="1" cellspacing="0" style="text-align: left;"
|-
| [[File:2014-svs-gw1.png|thumb|Spectrograms for GW1]]
| [[File:2014-svs-hkhs1.png|thumb|Spectrograms for HKHS1]]
| [[File:2014-svs-hkhs2.png|thumb|Spectrograms for HKHS2]]
| [[File:2014-svs-hkhs3.png|thumb|Spectrograms for HKHS3]]
|-
| [[File:2014-svs-iiy1.png|thumb|Spectrograms for IIY1]]
| [[File:2014-svs-iiy2.png|thumb|Spectrograms for IIY2]]
| [[File:2014-svs-jl1.png|thumb|Spectrograms for JL1]]
| [[File:2014-svs-lfr1.png|thumb|Spectrograms for LFR1]]
|-
| [[File:2014-svs-rna1.png|thumb|Spectrograms for RNA1]]
| [[File:2014-svs-rp1.png|thumb|Spectrograms for RP1]]
| [[File:2014-svs-yc1.png|thumb|Spectrograms for YC1]]
|}

=== Labels ===

'''a''' = input mixture x, '''b''' = ground truth voice for x, '''c''' = extracted voice from x, 
'''d''' = input mixture y, '''e''' = ground truth voice for y, '''f''' = extracted voice from y, 
'''g''' = input mixture z, '''h''' = ground truth voice for z, '''i''' = extracted voice from z 

== Runtime Data ==

<csv>2014/svs/runtime.csv</csv>

2014:Singing Voice Separation Results

2014-10-16T15:45:12Z

Tak-Shing Chan:

== Introduction ==

=== Description ===

These are the results for the 2014 running of the Singing Voice Separation task set. For more information about this task set please refer to the [[2014:Singing Voice Separation]] page.

=== Legend ===

{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Submission code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract PDF
! width="440" | Contributors
|-
! GW1
| Bayesian Singing-Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/GW1.pdf PDF] || Guan-Xiang Wang, Po-Kai Yang, Chung-Chien Hsu, Jen-Tzung Chien
|-
! HKHS1
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS1.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! HKHS2
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS2.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! HKHS3
| Singing-Voice Separation using Deep Recurrent Neural Networks || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/HKHS3.pdf PDF] || Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis
|-
! IIY1
| IIY1 || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/IIY1.pdf PDF] || Yukara Ikemiya
|-
! IIY2
| IIY2 || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/IIY2.pdf PDF] || Yukara Ikemiya
|-
! JL1
| MIREX 2014: Singing Voice Separation using Spectral/Temporal Discontinuity || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/JL1.pdf PDF] || Il-Young Jeong, Kyogu Lee
|-
! LFR1
| LFR1 || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/LFR1.pdf PDF] || Antoine Liutkus, Derry Fitzgerald, Zafar Rafii
|-
! RNA1
| Singing Voice Separation using Adaptive Window Harmonic Sinusoidal Modeling || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/RNA1.pdf PDF] || Preeti Rao, Nagesh Nayak, Sharath Adavanne
|-
! RP1
| REPET-SIM for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/RP1.pdf PDF] || Zafar Rafii, Bryan Pardo
|-
! YC1
| MIREX 2014 Submission for Singing Voice Separation || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/YC1.pdf PDF] || Frederick Yen, Tai-Shih Chi
|}

==== Evaluation Criteria ====

'''GNSDR''' = Global Normalized Signal-to-Distortion Ratio 
'''NSDR''' = Normalized Signal-to-Distortion Ratio 
'''SIR''' = Signal-to-Interference Ratio 
'''SAR''' = Signal-to-Artifacts Ratio 

== Summary ==

=== Summary Results ===

{| class="sortable" border="1" cellspacing="0" style="text-align: left;"
|- style="background: yellow;"
! Algorithm !! Voice GNSDR (dB) !! Music GNSDR (dB) !! Runtime (hh)
|-
| GW1 || 2.8861 || 5.2549 || 24
|-
| HKHS1 || -1.3988 || 0.3483 || 06
|-
| HKHS2 || -1.9413 || 0.5239 || 06
|-
| HKHS3 || -2.4807 || 0.1414 || 06
|-
| IIY1 || 4.2190 || 7.7893 || 02
|-
| IIY2 || 4.4764 || 7.8661 || 02
|-
| JL1 || 4.1564 || 5.6304 || 01
|-
| LFR1 || 0.6499 || 3.0867 || 03
|-
| RNA1 || 3.6915 || 7.3153 || 06
|-
| RP1 || 2.8602 || 5.0306 || 01
|-
| YC1 || -0.8202 || -3.1150 || 13
|}

== NSDR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/nsdr-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/nsdr-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-nsdr.png]]

== SIR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/sir-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/sir-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-sir.png]]

== SAR ==

=== For the Singing Voice (dB) ===

<csv>2014/svs/sar-voice.csv</csv>

=== For the Music Accompaniment (dB) ===

<csv>2014/svs/sar-music.csv</csv>

=== Boxplots ===

[[File:2014-svs-sar.png]]

== Individual Spectrograms ==

As the MIREX test set is private, we use three other songs with similar characteristics to demonstrate the algorithms.

{| border="1" cellspacing="0" style="text-align: left;"
|-
| [[File:2014-svs-gw1.png|thumb|Spectrograms for GW1]]
| [[File:2014-svs-hkhs1.png|thumb|Spectrograms for HKHS1]]
| [[File:2014-svs-hkhs2.png|thumb|Spectrograms for HKHS2]]
| [[File:2014-svs-hkhs3.png|thumb|Spectrograms for HKHS3]]
|-
| [[File:2014-svs-iiy1.png|thumb|Spectrograms for IIY1]]
| [[File:2014-svs-iiy2.png|thumb|Spectrograms for IIY2]]
| [[File:2014-svs-jl1.png|thumb|Spectrograms for JL1]]
| [[File:2014-svs-lfr1.png|thumb|Spectrograms for LFR1]]
|-
| [[File:2014-svs-rna1.png|thumb|Spectrograms for RNA1]]
| [[File:2014-svs-rp1.png|thumb|Spectrograms for RP1]]
| [[File:2014-svs-yc1.png|thumb|Spectrograms for YC1]]
|}

=== Labels ===

'''a''' = input mixture x, '''b''' = ground truth voice for x, '''c''' = extracted voice from x, 
'''d''' = input mixture y, '''e''' = ground truth voice for y, '''f''' = extracted voice from y, 
'''g''' = input mixture z, '''h''' = ground truth voice for z, '''i''' = extracted voice from z 

== Runtime Data ==

<csv>2014/svs/runtime.csv</csv>

File:2014-svs-yc1.png

2014-10-16T15:02:34Z

Tak-Shing Chan: YC1

YC1

File:2014-svs-rp1.png

2014-10-16T15:02:09Z

Tak-Shing Chan: RP1

RP1

File:2014-svs-rna1.png

2014-10-16T15:00:26Z

Tak-Shing Chan: RNA1

RNA1

File:2014-svs-lfr1.png

2014-10-16T14:59:47Z

Tak-Shing Chan: LFR1

LFR1

File:2014-svs-jl1.png

2014-10-16T14:59:17Z

Tak-Shing Chan: JL1

JL1

File:2014-svs-iiy2.png

2014-10-16T14:58:31Z

Tak-Shing Chan: IIY2

IIY2

File:2014-svs-iiy1.png

2014-10-16T14:57:44Z

Tak-Shing Chan: IIY1

IIY1

File:2014-svs-hkhs3.png

2014-10-16T14:56:45Z

Tak-Shing Chan: HKHS3

HKHS3

File:2014-svs-hkhs2.png

2014-10-16T14:56:04Z

Tak-Shing Chan: HKHS2

HKHS2