MIREX Wiki - User contributions [en]

2012:Audio Similarity 2012 Graders

2012-09-11T12:24:04Z

Liming: /* Sign Up Area */

=AMS 2012 Graders=

Welcome to the AMS grader sign-up page. Please give us your name and email contact information. If you obscure your email, please make it relatively obvious to us how to parse the address.

Template: Name. Location. <Email>

Sample: J. Stephen Downie. Illinois, USA. <jdownie@illinois.edu>

==Special Comments==
We are under the usual time constraints this year. ISMIR 2012 begins 8
October. We need to have all the MIREX results calculated and posted by
our 1 October target date (fingers crossed)!

We hope to open the Evalutron 6000 (E6K) v.2 grading system by
Friday, 14 September. To meet our 1 October goal, we must have all the
AMS and SMS similarity grades entered into the E6K by Wednesday, 26 September. So, if you are kind enough to sign up to be a grader, PLEASE understand
that we REALLY need you complete your assigned grading by 26 September.

If you are a SMS or AMS participant, we ask that you do what you can to
encourage adults over 18 years of age to be graders.

We are looking for 50 graders for AMS this year. If we make our quota of 50 graders, each grader will be responsible for two query lists. If we fall short, and get around 34 graders, we will be asking each grader to grade 3 queries. In this worst case scenario, we still expect the grading process to take between 2.5 to 3 hours (or less) for each grader.

This year the query lists seem to be moderate in length so tackling two queries lists should not be too onerous. For safety's sake, we would like to see say, an extra 5 or so names on the sign up sheet below. The "extra names" on the sign up sheet will be considered "back up" graders. We will assign grading tasks in the order of the names as they appear below.

=Sign Up Area=

# Minje Kim. Savoy, IL, USA. <minje.kim at gmail com>
# Jia-Min Ren, Tsing Hua Univ., Taiwan <jmren at mirlab.org>
# Nick Seaver, UC Irvine, CA, USA. <nseaver at uci.edu>
# Julián Urbano, Spain <jurbano at inf.uc3m.es>
# Carlos N. Silla Jr., Brazil <carlos.sillajr at gmail.com>
# Hannah Robertson. QC, Canada. <hannah.robertson at mail.mcgill.ca>
# Emmanouil Benetos, UK, <emmanouilb at eecs.qmul.ac.uk>
# Franz de Leon, UK, <fadl1d09 at ecs.soton.ac.uk>
# Dominik Schnitzer, A, <dominik.schnitzer at ofai.at>
# Arthur Flexer, A, <arthur.flexer at ofai.at>
# Jan Schlueter, A, <jan.schlueter at ofai.at>
# Aggelos Gkiokas, Greece, <agkiokas at ilsp.gr>
# George Tzanetakis, Canada <gtzan at cs.uvic.ca>
# Antonio Deusany de Carvalho Junior, USP, Brazil <dj at ime.usp.br>
# Peter Knees, Austria <peter.knees at jku.at>
# Ming Li, IOACAS, Beijing China, <liming.ioa at gmail.com>

2011:Audio Similarity 2011 Graders

2011-09-30T01:35:47Z

Liming: /* Sign Up Area */

=AMS 2011 Graders=

Welcome to the AMS grader sign-up page. Please give us your name and email contact information. If you obscure your email, please make it relatively obvious to us how to parse the address.

Template: Name. Location. <Email>

Sample: J. Stephen Downie. Illinois, USA. <jdownie@illinois.edu>

==Special Comments==
We are under time constraints this year because ISMIR 2011
begins Monday, 24th of October. We need to have all the final
results calculated and posted by the 14th of October target date (fingers crossed).

We hope to open the the Evalutron 6000 (E6K) v.2 grading system by Friday, 30th
September. To meet our goal, we must have all the AMS and SMS
similarity grades entered into the E6K by Wednesday, Oct. 12th. So, if you are kind enough to sign up to be a grader, please understand that we really need you complete your assigned grading
by Wednesday, Oct. 12th.

If you are a SMS or AMS participant, we ask that you do what you can to
encourage adults over 18 years of age to be graders.

We are looking for 50 graders for AMS this year. If we make our quota of 50 graders, each grader will be responsible for two query lists. If we fall short, and get around 34 graders, we will be asking each grader to grade 3 queries. In this worst case scenario, we still expect the grading process to take between 2.5 to 3 hours (or less) for each grader.

This year the query lists seem to be moderate in length so tackling two queries lists should not be too onerous. For safety's sake, we would like to see say, an extra 5 or so names on the sign up sheet below. The "extra names" on the sign up sheet will be considered "back up" graders. We will assign grading tasks in the order of the names as they appear below.

=Sign Up Area=

# Brian McFee. California, USA. <bmcfee@cs.ucsd.edu>
# Steve Tjoa. San Francisco, CA, USA. <steve at imagine-research com>
# Jia-Min Ren. Hsinchu, Taiwan. <jmren at mirlab org>
# Sally Jo Cunningham, Hamilton, New Zealand. <sallyjo@cs.waikato.ac.nz>
# Sungkyun Chang. Suwon, Korea. <rayno1 at snu.ac.kr>
# Yin-Tzu Lin. Taipei, Taiwan. <known at cmlab.csie.ntu.edu.tw>
# Franz de Leon. Southampton, UK. <fadl1d09@ecs.soton.ac.uk>
# Simone Sammartino. Málaga, Spain. <ssammartino@ic.uma.es>
# Arthur Flexer, OFAI, Austria <arthur.flexer at ofai.at>
# Dominik Schnitzer, OFAI, Austria <dominik.schnitzer at ofai.at>
# Jan Schlueter, OFAI, Austria <jan.schlueter at ofai.at>
# Cristina de la Bandera. Málaga, Spain. <cdelabandera@ic.uma.es>
# Bart Stasiak. Lodz, Poland. <basta -@- ics.p.lodz.pl>
# Thierry Bertin-Mahieux. New York, USA. <tb2332@columbia.edu>
# Benjamin Martin. Bordeaux, France. <benjamin.martin@labri.fr>
# Ruofeng Chen. Georgia, USA. <ruofengchen (at) gatech (dot) edu>
# Bo Xie. Atlanta, USA. <bo.xie (at) gatech (dot) edu>
# Chung-Che Wang. Hsinchu, Taiwan. <geniusturtle (at) mirlab dot org>
# Peter Knees, cp.jku, Austria. <peter.knees (at) jku.at>
# Markus Schedl, cp.jku, Austria <markus.schedl (at) jku.at>
# Audrey Laplante. Montréal, Canada. <audrey.laplante (at) umontreal.ca>
# Ajay Ramaseshan. Espoo, Finland. <ajayram (at) cis.hut.fi>
# Matt Hoffman. New York, New York, USA. <mdhoffma@cs.princeton.edu>
# Ming Li. Beijing, China. <liming.ioa (at) gmail dot com>
# Molly McLaughlin. Champaign, IL, USA. <mclaugh@illinois.edu>
# Nick Seaver, Irvine, CA, USA. <nseaver at uci dot edu>
# Polina Proutskova, Goldsmiths, London, <proutskova (at) googlemail.com>
# Deshun Yang, Beijing, P. R. China. <sidney172 (at) gmail.com>
# Minje Kim, Urbana, IL, USA <minje AT illinois DOT edu>
# Charlie Inskip, London, Eng <charlesinskip AT hotmail DOT com>
# Emmanouil Benetos, London, UK <emmanouilb (at) eecs.qmul.ac.uk>
# Garret Gengler. Champaign, IL, USA. <garretg (at) illinois.edu>
# Ioanna Filippidi. Greece, GR, <filippidioanna (at) gmail.com>
# Bruno Rocha. Braga, Portugal. <brunomachadorocha (at) gmail.com>
# Matt Voglewede. Eugene, OR, USA. <mjv at uoregon edu>
# Dongying Zhang, Beijing, China <zhangdongying1986@gmail.com>
# Claire Bradford. London, UK. <claire (at) wonderland.org>
# Kahyun Choi, Urbana, IL, USA <ckahyu2 AT illinois DOT edu>
# Maria Theresa de Leon. Southampton, UK. <mtdl1c09@ecs.soton.ac.uk>
# Jacob Jett. Champaign, IL, USA. <jjett2@illinois.edu>
# Fu-Hai Frank Wu. Hsinchu, Taiwan. <frankwu@mirlab.org>

2011:Audio Similarity 2011 Graders

2011-09-28T06:09:20Z

Liming: /* Sign Up Area */

2011:Audio Similarity 2011 Graders

2011-09-28T06:08:17Z

Liming: /* Sign Up Area */

MIREX Impact Statements

2011-05-12T13:43:05Z

Liming: /* Impact Statements */

=Introduction=

Dear MIR Community:

We are gathering evidence of impact for MIREX. We would appreciate it greatly if you would post comments about how MIREX has impacted or influenced your work directly or MIR research in general.

In a similar vein, we are also gathering citations to MIREX-related papers as further evidence of impact for MIREX on the [[MIREX Impact Citations]] page.

We are collecting these statements and citations as evidence influence and success to submit to funding agencies, research administrators and/or future collaborators.

Thank you very much.

J. Stephen Downie 
Director, IMIRSEL 

=Impact Statements=

I am a researcher in audio signal processing. Although music information retrieval is not my main focus, I have been following the MIREX campaigns from the very beginning as one of the most exciting initiatives in the field. What makes MIREX unique is the attention made to the specification of the datasets and the evaluation procedures by the entrants themselves. I soon realized that this is the best way to attract a large number of entrants and evaluate tasks that people really care about. Inspired by this view, I founded an annual evaluation campaign relating to my main focus (audio source separation), which has attracted more than 80 entrants in total and changed the view of the problem. See E. Vincent, S. Araki et al., "The Signal Separation Evaluation Campaign (2007-2010): Achievements and Remaining Challenges", Technical Report no. RR-7581, INRIA, 2011 [http://hal.inria.fr/inria-00579398/PDF/RR-7581.pdf]. The funding application made by IMIRSEL aims to explore business models for MIREX and result in the creation of a sustainable MIREX model. This issue is also crucial for SiSEC and other medium-scale campaigns which cannot benefit from a large-scale organization such as TREC. IMIRSEL has already paved the way via their online DIY-system. I have no doubt that the provided funding will enable them to provide a conclusive answer this issue, which will be mostly reusable in SiSEC and other campaigns.
Emmanuel Vincent, Research Scientist, INRIA, France.

"The evaluation of cover song identification and similarity systems is a complex task, and it is difficult to find in the literature a common methodology for that. The only existing attempt to compare version identification systems is found in the Music Information Retrieval Evaluation eXchange (MIREX) initiative ..." in J. Serrà, E. Gómez and P. Herrera. Audio cover song identification and similarity: background, approaches, evaluation, and beyond. In Advances in Music Information Retrieval, Z. W. Ras and A. A. Wieczorkowska editors. Studies in Computational Intelligence series, Springer-Verlag Berlin / Heidelberg, vol. 274, ch. 14, pp. 307-332. March 2010. http://mtg.upf.edu/node/1389

"The MIR evaluation exchange (MIREX) is an international community-based framework for the formal evaluation of MIR systems and algorithms [21]. Among other tasks, MIREX allows comparing different algorithms for artist identification, genre classification, or music transcriptionNote2 . In particular, MIREX allows for an objective assessment of the accuracy of different cover song identification algorithms. ..." in J. Serrà, X. Serra and R. G. Andrzejak. Cross recurrence quantification for cover song identification. New Journal of Physics, vol. 11, art. 093017. September 2009. http://mtg.upf.edu/node/1390

"MIREX is the widest participation contest in music information retrieval (MIR) field in the world. This contest holds annually and proposes new competitive tasks every year to keep up with the change in MIR field. The contest includes most research topics in MIR filed from traditional tasks such as Query by Singing/Humming task and Audio Genre Classification task to newest tasks such as Audio Music Mood Classification and Audio Tag Classification." Research News at Institute of Acoustics, Chinese Academy of Sciences, Nov. 2009, http://english.ioa.cas.cn/rh/as/200911/t20091106_46842.html

2010:Audio Similarity 2010 Graders

2010-07-14T09:25:24Z

Liming: /* Sign Up Area */

=AMS 2010 Graders=

Welcome to the AMS grader sign-up page. Please give us your name and email contact information. If you obscure your email, please make it relatively obvious to us how to parse the address.

Template: Name. Location. <Email>

Sample: J. Stephen Downie. Illinois, USA. <jdownie@illinois.edu>

=Special Comments=
We are under extraordinary time constraints this year because ISMIR 2010
is being held starting 9 August and we need to have all the final
results calculated and posted by our 2 August target date (fingers crossed).

We hope to open the the Evalutron 6000 (E6K) v.2 grading system by 20
July. To meet our 2 August goal, we must have all the AMS and SMS
similarity grades entered into the E6K by 27 July. YES, THIS GIVES US
ONLY ONE WEEK! So, if you are kind enough to sign up to be a grader,
please understand that we really need you complete your assigned grading
by 27 July.

If you are a SMS or AMS participant, we ask that you do what you can to
encourage adults over 18 years of age to be graders.

We are looking for 50 graders for AMS this year. Each grader will be responsible for two query lists. This year the query lists seem to be moderate in length so tackling two queries lists should not be too onerous. For safety's sake, we would like to see say, an extra five or so names on the sign up sheet below. The "extra names" on the sign up sheet will be considered "back up" graders. We will assign grading tasks in the order of the names as they appear below.

=Sign Up Area=
Martin Ariel Hartmann. Buenos Aires, Argentina. <martin.hartmann@jyu.fi>

Sally Jo Cunningham. Hamilton, New Zealand. <sallyjo@cs.waikato.ac.nz>

Matt Hoffman. New York, USA. <mdhoffma@cs.princeton.edu>

Thierry Bertin-Mahieux. New York, USA. <tb2332@columbia.edu>

Dominik Schnitzer, Vienna, Austria <dominik.schnitzer (dumdidum) ofai.at>

Ming Li, Beijing, China, <liming . ioa [AT] gmail dot com>