2006:Evalutron6000 Issues

Introduction

Standby for an explication of this page and the Evalutron6000 setup for the Audio Similarity and Retrieval and Symbolic Melodic Similarity tasks.

AudioSim Assumptions

As of 22 August 2006 here is a quick sketch of the parameter space that we are working under (subject to realities):

# of votes/eyes per candidate: 5
# of alogrithms evaluated: 6
# of candidates per query per algo: 5
# of queries: 30
# of comparisons: 4500
# of "graders": 20
# of comparisons per "grader": 225
# of seconds assumed per comparison: 45
# of total seconds to grade per grader: 10,125
# of hours to grade per grader: 2.812

As you can see, change one value, and the rest re-jiggers itself (sometimes in very nasty ways).

Potential Audio Simularity "Graders"

• J. Stephen Downie, IMIRSEL
• Andreas Ehmann, IMIRSEL
• Kris West, IMIRSEL
• Xiao Hu, IMIRSEL
• Mert Bay, IMIRSEL
• M. Cameron Jones, IMIRSEL
• Paul Lamere, IMIRSEL
• Martin McCrory, IMIRSEL
• Qin Wei, IMIRSEL
• Beth Logan
• Mark Levy, QMUL

testing