Difference between revisions of "2008:Multiple Fundamental Frequency Estimation & Tracking Results"

From MIREX Wiki
Line 5: Line 5:
 
====Team ID====
 
====Team ID====
  
−
'''CL1''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf C. Cao, M. Li 1]<br />
+
'''CC1''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf C. Cao, M. Li 1]<br />
−
'''CL2''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf C. Cao, M. Li 2]<br />
+
'''CC2''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf C. Cao, M. Li 2]<br />
 
'''DRD''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf J-L. Durrieu, G. Richard, B. David]<br />
 
'''DRD''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf J-L. Durrieu, G. Richard, B. David]<br />
 
'''EOS''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf K. Egashira, N. Ono, S. Sagayama]<br />
 
'''EOS''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf K. Egashira, N. Ono, S. Sagayama]<br />
Line 21: Line 21:
 
'''ZR2''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf R. Zhou, J. D. Reiss 2]<br />
 
'''ZR2''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf R. Zhou, J. D. Reiss 2]<br />
 
'''ZR3''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf R. Zhou, J. D. Reiss 3]<br />
 
'''ZR3''' = [https://www.music-ir.org/mirex/2008/abs/XXX.pdf R. Zhou, J. D. Reiss 3]<br />
 +
 +
===Overall Summary Results Task1===
 +
Below are the average scores across 36 test files. These files consisted of 9 groups, each group having 4 files ranging from 2 polyphony to 5 polyphony. 28 real recordings, 8 synthesized from RWC samples.
 +
 +
<csv>multif0/task1_summary.csv</csv>
 +
 +
====Detailed Results====
 +
 +
<csv>multif0/task1_res.csv</csv>
 +
 +
====Detailed Chroma Results====
 +
Here, accuracy is assessed on chroma results (i.e. all F0's are mapped to a single octave before evaluating)
 +
 +
<csv>multif0/task1_chroma.csv</csv>
 +
 +
===Individual Results Files for Task 1===
 +
====Individual results: Scores per Query====
 +
'''CC1''' = [https://www.music-ir.org/mirex/2008/results/multif0/CC1task1.tar.gz C. Cao, M. Li 1]<br />
 +
'''CC2''' = [https://www.music-ir.org/mirex/2008/results/multif0/CC2task1.tar.gz C. Cao, M. Li 2]<br/>
 +
'''DRD''' = [https://www.music-ir.org/mirex/2008/results/multif0/DRDtask1.tar.gz J-L. Durrieu, G. Richard, B. David] <br/>
 +
'''EBD1''' = [https://www.music-ir.org/mirex/2008/results/multif0/EBD1task1.tar.gz CV. Emiya, R. Badeau, B. David 1]<br />
 +
'''EBD2''' = [https://www.music-ir.org/mirex/2008/results/multif0/EBD2task1.tar.gz V. Emiya, R. Badeau, B. David 2]<br />
 +
'''EOS''' = [https://www.music-ir.org/mirex/2008/results/multif0/EOStask1.tar.gz K. Egashira, N. Ono, S. Sagayama]<br />
 +
'''MG''' = [https://www.music-ir.org/mirex/2008/results/multif0/MGtask1.tar.gz M. Groble]<br />
 +
'''PI''' = [https://www.music-ir.org/mirex/2008/results/multif0/PItask1.tar.gz A. Pertusa, J. M. I├▒esta]<br />
 +
'''RFF1''' = [https://www.music-ir.org/mirex/2008/results/multif0/RFF1task1.tar.gz G. Reis, F. Fernandez, A. Ferreira 1]<br />
 +
'''RFF2''' = [https://www.music-ir.org/mirex/2008/results/multif0/RFF2task1.tar.gz GG. Reis, F. Fernandez, A. Ferreira 2]<br />
 +
'''RK''' = [https://www.music-ir.org/mirex/2008/results/multif0/RKtask1.tar.gz M. Ryynänen, A. Klapuri]<br />
 +
'''VBB''' = [https://www.music-ir.org/mirex/2008/results/multif0/VBBtask1.tar.gz E. Vincent, N. Bertin, R. Badeau]<br />
 +
'''YRC''' = [https://www.music-ir.org/mirex/2008/results/multif0/YRCtask1.tar.gz C. Yeh, A. Roebel, W-C. Chang]<br />
 +
 +
 +
=====Info about the filenames=====
 +
The filenames starting with part* comes from acoustic woodwind recording, the ones starting with RWC are synthesized. The legend about the instruments are:
 +
 +
'''bs''' = bassoon
 +
 +
'''cl''' = clarinet
 +
 +
'''fl''' = flute
 +
 +
'''hn''' = horn
 +
 +
'''ob''' = oboe
 +
 +
'''vl''' = violin
 +
 +
'''cel''' = cello
 +
 +
'''gtr''' = guitar
 +
 +
'''sax''' saxophone
 +
 +
'''bass''' = electric bass guitar
 +
 +
====Run Times====
 +
<csv>multiF0_task1_runtimes.csv</csv>
 +
 +
===Overall Summary Results Task II===
 +
This subtask is evaluated in two different ways. In the first setup , a returned note is assumed correct if its onset is within +-50ms of a ref note and its F0 is within +- quarter tone of the corresponding reference note, ignoring the returned offset values. In the second setup, on top of the above requirements, a correct returned note is required to have an offset value within 20% of the ref notes duration around the ref note`s offset, or within 50ms whichever is larger.
 +
 +
A total of 30 files were used in this task: 16 real recordings, 8 synthesized from RWC samples, and 6 piano. The results below are the average of these 30 files.
 +
 +
===Results Based on Onset Only===
 +
<csv>multiF0.note.onset.only.eval.csv</csv>
 +
 +
===Results Based on Onset-Offset===
 +
<csv>multiF0.note.eval.csv</csv>
 +
 +
===Piano Results based on Onset Only===
 +
<csv>multiF0.note.onset.only.eval.for.piano.csv</csv>
 +
 +
==Individual Results Files for Task 2==
 +
 +
===Info About Filenames===
 +
The filenames starting with part* comes from acoustic woodwind recording, the ones starting with RWC are synthesized. The piano files are: RA_C030_align.wav,bach_847TESTp.wav,beet_pathetique_3TESTp.wav,mz_333_1TESTp.wav,scn_4TESTp.wav.note, ty_januarTESTp.wav.note
 +
 +
===Run Times===
 +
<csv>multiF0task2.runtimes.csv</csv>
 +
  
 
[[Category: Results]]
 
[[Category: Results]]

Revision as of 14:06, 11 September 2008

Introduction

These are the results for the 2008 running of the Multiple Fundamental Frequency Estimation and Tracking task. For background information about this task set please refer to the Multiple Fundamental Frequency Estimation & Tracking page.

General Legend

Team ID

CC1 = C. Cao, M. Li 1
CC2 = C. Cao, M. Li 2
DRD = J-L. Durrieu, G. Richard, B. David
EOS = K. Egashira, N. Ono, S. Sagayama
EBD1 = V. Emiya, R. Badeau, B. David 1
EBD2 = V. Emiya, R. Badeau, B. David 2
MG = M. Groble
PI = A. Pertusa, J. M. I├▒esta
RFF1 = G. Reis, F. Fernandez, A. Ferreira 1
RFF2 = G. Reis, F. Fernandez, A. Ferreira 2
RK = M. Ryynänen, A. Klapuri
VBB = E. Vincent, N. Bertin, R. Badeau
YRC = C. Yeh, A. Roebel, W-C. Chang
ZR1 = R. Zhou, J. D. Reiss 1
ZR2 = R. Zhou, J. D. Reiss 2
ZR3 = R. Zhou, J. D. Reiss 3

Overall Summary Results Task1

Below are the average scores across 36 test files. These files consisted of 9 groups, each group having 4 files ranging from 2 polyphony to 5 polyphony. 28 real recordings, 8 synthesized from RWC samples.

file /nema-raid/www/mirex/results/multif0/task1_summary.csv not found

Detailed Results

file /nema-raid/www/mirex/results/multif0/task1_res.csv not found

Detailed Chroma Results

Here, accuracy is assessed on chroma results (i.e. all F0's are mapped to a single octave before evaluating)

file /nema-raid/www/mirex/results/multif0/task1_chroma.csv not found

Individual Results Files for Task 1

Individual results: Scores per Query

CC1 = C. Cao, M. Li 1
CC2 = C. Cao, M. Li 2
DRD = J-L. Durrieu, G. Richard, B. David
EBD1 = CV. Emiya, R. Badeau, B. David 1
EBD2 = V. Emiya, R. Badeau, B. David 2
EOS = K. Egashira, N. Ono, S. Sagayama
MG = M. Groble
PI = A. Pertusa, J. M. I├▒esta
RFF1 = G. Reis, F. Fernandez, A. Ferreira 1
RFF2 = GG. Reis, F. Fernandez, A. Ferreira 2
RK = M. Ryynänen, A. Klapuri
VBB = E. Vincent, N. Bertin, R. Badeau
YRC = C. Yeh, A. Roebel, W-C. Chang


Info about the filenames

The filenames starting with part* comes from acoustic woodwind recording, the ones starting with RWC are synthesized. The legend about the instruments are:

bs = bassoon

cl = clarinet

fl = flute

hn = horn

ob = oboe

vl = violin

cel = cello

gtr = guitar

sax saxophone

bass = electric bass guitar

Run Times

file /nema-raid/www/mirex/results/multiF0_task1_runtimes.csv not found

Overall Summary Results Task II

This subtask is evaluated in two different ways. In the first setup , a returned note is assumed correct if its onset is within +-50ms of a ref note and its F0 is within +- quarter tone of the corresponding reference note, ignoring the returned offset values. In the second setup, on top of the above requirements, a correct returned note is required to have an offset value within 20% of the ref notes duration around the ref note`s offset, or within 50ms whichever is larger.

A total of 30 files were used in this task: 16 real recordings, 8 synthesized from RWC samples, and 6 piano. The results below are the average of these 30 files.

Results Based on Onset Only

file /nema-raid/www/mirex/results/multiF0.note.onset.only.eval.csv not found

Results Based on Onset-Offset

file /nema-raid/www/mirex/results/multiF0.note.eval.csv not found

Piano Results based on Onset Only

file /nema-raid/www/mirex/results/multiF0.note.onset.only.eval.for.piano.csv not found

Individual Results Files for Task 2

Info About Filenames

The filenames starting with part* comes from acoustic woodwind recording, the ones starting with RWC are synthesized. The piano files are: RA_C030_align.wav,bach_847TESTp.wav,beet_pathetique_3TESTp.wav,mz_333_1TESTp.wav,scn_4TESTp.wav.note, ty_januarTESTp.wav.note

Run Times

file /nema-raid/www/mirex/results/multiF0task2.runtimes.csv not found

Retrieved from "https://music-ir.org/mirex/w/index.php?title=2008:Multiple_Fundamental_Frequency_Estimation_%26_Tracking_Results&oldid=4247"