View source for 2013:Multiple Fundamental Frequency Estimation & Tracking Results

==Introduction==
These are the results for the 2008 running of the Multiple Fundamental Frequency Estimation and Tracking task. For background information about this task set please refer to the [[2013:Multiple Fundamental Frequency Estimation & Tracking]] page.

==General Legend==
{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
	|- style="background: yellow;"
	! width="80" | Sub code 
	! width="200" | Submission name 
	! width="80" style="text-align: center;" | Abstract 
	! width="440" | Contributors
	|-
	! CDM1
	| CDM1  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/CDM1.pdf PDF] || [http://c4dm.eecs.qmul.ac.uk/ Tian Cheng], [http://www.eecs.qmul.ac.uk/~simond Simon Dixon], [http://c4dm.eecs.qmul.ac.uk/ Matthias Mauch]
	|-
	! CDM2
	| CDM2  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/CDM2.pdf PDF] || [http://c4dm.eecs.qmul.ac.uk/ Tian Cheng], [http://www.eecs.qmul.ac.uk/~simond Simon Dixon], [http://c4dm.eecs.qmul.ac.uk/ Matthias Mauch]
	|-
	! CDM3
	| CDM3  ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/CDM3.pdf PDF] || [http://c4dm.eecs.qmul.ac.uk/ Tian Cheng], [http://www.eecs.qmul.ac.uk/~simond Simon Dixon], [http://c4dm.eecs.qmul.ac.uk/ Matthias Mauch]
	|-
	! BW1
	| MSSIPLCA_fast_MultiF0   ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/BW1.pdf PDF] || [http://www.soi.city.ac.uk/~sbbj660/ Emmanouil Benetos], [http://www.soi.city.ac.uk/~sa746/  Tillman Weyde]
	|-
	! BW2
	| MSSIPLCA_fast_NoteTracking1   ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/BW2.pdf PDF] || [http://www.soi.city.ac.uk/~sbbj660/ Emmanouil Benetos], [http://www.soi.city.ac.uk/~sa746/  Tillman Weyde]
	|-
	! BW3
	| MSSIPLCA_fast_NoteTracking2   ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/BW3.pdf PDF] || [http://www.soi.city.ac.uk/~sbbj660/ Emmanouil Benetos], [http://www.soi.city.ac.uk/~sa746/  Tillman Weyde]
	|-
        |}

==Task 1: Multiple Fundamental Frequency Estimation (MF0E)==

===MF0E Overall Summary Results===
Below are the average scores across 40 test files. These files come from 3 different sources: woodwind quintet recording of bassoon, clarinet, horn,flute and oboe (UIUC); Rendered MIDI using RWC database donated by IRCAM and a quartet recording of bassoon, clarinet,  violin and sax  donated by Dr. Bryan Pardo`s Interactive Audio Lab (IAL). 20 files coming from 5 sections of the woodwind recording where each section has 4 files ranging from 2 polyphony to 5 polyphony. 12 files from IAL, coming from 4 different songs ranging from 2 polyphony to 4 polyphony and 8 files from RWC synthesized midi ranging from 2 different songs ranging from 2 polphony to 5 polyphony. 

<csv>2013/mf0/est/summary/task1.overall.csv</csv> 

====Detailed Results====

<csv>2013/mf0/est/summary/task1.results.csv</csv>

====Detailed Chroma Results====
Here, accuracy is assessed on chroma results (i.e. all F0's are mapped to a single octave before evaluating)

<csv>2013/mf0/est/summary/task1.chroma.results.csv</csv>

===Individual Results Files for Task 1===
'''BW1'''= [https://music-ir.org/mirex/results/2013/mf0/est/BW1.results.tar.gz Emmanouil Benetos, Tillman Weyde]<br />
'''CDM1'''= [https://music-ir.org/mirex/results/2013/mf0/est/CDM1.results.tar.gz Tian Cheng, Simon Dixon, Matthias Mauch]<br />
'''CDM2'''= [https://music-ir.org/mirex/results/2013/mf0/est/CDM2.results.tar.gz Tian Cheng, Simon Dixon, Matthias Mauch]<br />


=====Info about the filenames=====
The filenames starting with part* comes from acoustic woodwind recording, the ones starting with RWC are synthesized. The legend about the instruments are:

'''bs''' = bassoon,
'''cl''' = clarinet,
'''fl''' = flute,
'''hn''' = horn,
'''ob''' = oboe,
'''vl''' = violin,
'''cel''' = cello,
'''gtr''' = guitar,
'''sax''' = saxophone,
'''bass''' = electric bass guitar

===Run Times===

<csv>2013/mf0/est/runtimes_mf0_2013.csv</csv>

===Friedman tests for  Multiple Fundamental Frequency Estimation (MF0E)===
The Friedman test was run in MATLAB to test significant differences amongst systems with regard to the performance (accuracy) on individual files.

====Tukey-Kramer HSD Multi-Comparison====

<csv>2013/mf0/est/summary/Accuracy_Per_Song_Friedman_Mean_Rankstask1.friedman.Friedman_TukeyKramerHSD.csv</csv>

[[File:2013_Accuracy_Per_Song_Friedman_Mean_Rankstask1.friedman.Friedman_Mean_Ranks.png|500px]]

==Task 2:Note Tracking (NT)==
===NT Mixed Set Overall Summary Results===
This subtask is evaluated in two different ways. In the first setup , a returned note is assumed correct if its onset is within +-50ms of a ref note and its F0 is within +- quarter tone of the corresponding reference note, ignoring the returned offset values. In the second setup, on top of the above requirements, a correct returned note is required to have an offset value within 20% of the ref notes duration around the ref note`s offset, or within 50ms whichever is larger.

A total of 34 files were used in this subtask: 16 from woodwind recording, 8 from IAL quintet recording and 6 piano.  

<csv>2013/mf0/nt/summary/task2.overall.results.csv</csv>

====Detailed Results====

<csv>2013/mf0/nt/summary/task2.results.csv</csv>

====Detailed Chroma Results====
Here, accuracy is assessed on chroma results (i.e. all F0's are mapped to a single octave before evaluating)

<csv>2013/mf0/nt/summary/task2.chroma.results.csv</csv>


====Results Based on Onset Only====

<csv>2013/mf0/nt/summary/task2.onsetOnly.results.csv</csv>

====Chroma Results Based on Onset Only====

<csv>2013/mf0/nt/summary/task2.onsetOnly.chroma.results.csv</csv>

====Run Times====

<csv>2013/mf0/nt/runtimes_NT_2013.csv</csv>

====Friedman Tests for  Note Tracking====
The Friedman test was run in MATLAB to test significant differences amongst systems with regard to the F-measure  on individual files.

=====Tukey-Kramer HSD Multi-Comparison for Task2=====

<csv>2013/mf0/nt/summary/Accuracy_Per_Song_Friedman_Mean_Rankstask2.onsetOnly.friedman.Friedman_TukeyKramerHSD.csv</csv>

[[File:2013_Accuracy_Per_Song_Friedman_Mean_Rankstask2.onsetOnly.friedman.Friedman_Mean_Ranks.png|500px]]

===NT Piano-Only Overall Summary Results===
This subtask is evaluated in two different ways. In the first setup , a returned note is assumed correct if its onset is within +-50ms of a ref note and its F0 is within +- quarter tone of the corresponding reference note, ignoring the returned offset values. In the second setup, on top of the above requirements, a correct returned note is required to have an offset value within 20% of the ref notes duration around the ref note`s offset, or within 50ms whichever is larger.
6 piano recordings are evaluated separately for this subtask.

<csv>2013/mf0/nt/summary_piano_subtask/task2_piano.overall.results.csv</csv>

====Detailed Results====

<csv>2013/mf0/nt/summary_piano_subtask/task2_piano.results.csv</csv>

====Detailed Chroma Results====
Here, accuracy is assessed on chroma results (i.e. all F0's are mapped to a single octave before evaluating)

<csv>2013/mf0/nt/summary_piano_subtask/task2_piano.chroma.results.csv</csv>

====Results Based on Onset Only====

<csv>2013/mf0/nt/summary_piano_subtask/task2_piano.onsetOnly.results.csv</csv>

====Chroma Results Based on Onset Only====

<csv>2013/mf0/nt/summary_piano_subtask/task2_piano.onsetOnly.chroma.results.csv</csv>

===Individual Results Files for Task 2===
'''BW2'''= [https://music-ir.org/mirex/results/2013/mf0/nt/BW2.results.tar.gz Emmanouil Benetos, Tillman Weyde]<br />
'''BW3'''= [https://music-ir.org/mirex/results/2013/mf0/nt/BW3.results.tar.gz Emmanouil Benetos, Tillman Weyde]<br />
'''CDM3'''= [https://music-ir.org/mirex/results/2013/mf0/nt/CDM3.results.tar.gz Tian Cheng, Simon Dixon, Matthias Mauch]<br />