Difference between revisions of "2006:Evalutron6000 Issues"
From MIREX Wiki
CameronJones (talk | contribs) |
|||
| Line 1: | Line 1: | ||
=Introduction= | =Introduction= | ||
Standby for an explication of this page and the Evalutron6000 setup for the Audio Similarity and Retrieval and Symbolic Melodic Similarity tasks. | Standby for an explication of this page and the Evalutron6000 setup for the Audio Similarity and Retrieval and Symbolic Melodic Similarity tasks. | ||
| + | |||
| + | |||
| + | ==AudioSim Assumptions== | ||
| + | As of 22 August 2006 here is a quick sketch of the parameter space that we are working under (subject to realities): | ||
| + | <pre> | ||
| + | # of votes/eyes per candidate: 5 | ||
| + | # of alogrithms evaluated: 6 | ||
| + | # of candidates per query per algo: 5 | ||
| + | # of queries: 30 | ||
| + | # of comparisons: 4500 | ||
| + | # of "graders": 20 | ||
| + | # of comparisons per "grader": 225 | ||
| + | # of seconds assumed per comparison: 45 | ||
| + | # of total seconds to grade per grader: 10,125 | ||
| + | # of hours to grade per grader: 2.812 | ||
| + | </pre> | ||
| + | As you can see, change one value, and the rest re-jiggers itself (sometimes in very nasty ways). | ||
==Potential Audio Simularity "Graders"== | ==Potential Audio Simularity "Graders"== | ||
Revision as of 09:38, 22 August 2006
Introduction
Standby for an explication of this page and the Evalutron6000 setup for the Audio Similarity and Retrieval and Symbolic Melodic Similarity tasks.
AudioSim Assumptions
As of 22 August 2006 here is a quick sketch of the parameter space that we are working under (subject to realities):
# of votes/eyes per candidate: 5 # of alogrithms evaluated: 6 # of candidates per query per algo: 5 # of queries: 30 # of comparisons: 4500 # of "graders": 20 # of comparisons per "grader": 225 # of seconds assumed per comparison: 45 # of total seconds to grade per grader: 10,125 # of hours to grade per grader: 2.812
As you can see, change one value, and the rest re-jiggers itself (sometimes in very nasty ways).
Potential Audio Simularity "Graders"
- J. Stephen Downie, IMIRSEL
- Andreas Ehmann, IMIRSEL
- Kris West, IMIRSEL
- Xiao Hu, IMIRSEL
- Mert Bay, IMIRSEL
- M. Cameron Jones, IMIRSEL
- Paul Lamere, IMIRSEL
- Martin McCrory, IMIRSEL
- Qin Wei, IMIRSEL
- Beth Logan
- Mark Levy, QMUL
testing
