MIREX Impact Statements

Introduction

Dear MIR Community:

We are gathering evidence of impact for MIREX. We would appreciate it greatly if you would post comments about how MIREX has impacted or influenced your work directly or MIR research in general.

In a similar vein, we are also gathering citations to MIREX-related papers as further evidence of impact for MIREX on the MIREX Impact Citations page.

We are collecting these statements and citations as evidence influence and success to submit to funding agencies, research administrators and/or future collaborators.

Thank you very much.

J. Stephen Downie
Director, IMIRSEL

Impact Statements

I am a researcher in audio signal processing. Although music information retrieval is not my main focus, I have been following the MIREX campaigns from the very beginning as one of the most exciting initiatives in the field. What makes MIREX unique is the attention made to the specification of the datasets and the evaluation procedures by the entrants themselves. I soon realized that this is the best way to attract a large number of entrants and evaluate tasks that people really care about. Inspired by this view, I founded an annual evaluation campaign relating to my main focus (audio source separation), which has attracted more than 80 entrants in total and changed the view of the problem. See E. Vincent, S. Araki et al., "The Signal Separation Evaluation Campaign (2007-2010): Achievements and Remaining Challenges", Technical Report no. RR-7581, INRIA, 2011 [1]. The funding application made by IMIRSEL aims to explore business models for MIREX and result in the creation of a sustainable MIREX model. This issue is also crucial for SiSEC and other medium-scale campaigns which cannot benefit from a large-scale organization such as TREC. IMIRSEL has already paved the way via their online DIY-system. I have no doubt that the provided funding will enable them to provide a conclusive answer this issue, which will be mostly reusable in SiSEC and other campaigns. Emmanuel Vincent, Research Scientist, INRIA, France.

"The evaluation of cover song identification and similarity systems is a complex task, and it is difficult to find in the literature a common methodology for that. The only existing attempt to compare version identification systems is found in the Music Information Retrieval Evaluation eXchange (MIREX) initiative ..." in J. Serrà, E. Gómez and P. Herrera. Audio cover song identification and similarity: background, approaches, evaluation, and beyond. In Advances in Music Information Retrieval, Z. W. Ras and A. A. Wieczorkowska editors. Studies in Computational Intelligence series, Springer-Verlag Berlin / Heidelberg, vol. 274, ch. 14, pp. 307-332. March 2010. http://mtg.upf.edu/node/1389

"The MIR evaluation exchange (MIREX) is an international community-based framework for the formal evaluation of MIR systems and algorithms [21]. Among other tasks, MIREX allows comparing different algorithms for artist identification, genre classification, or music transcriptionNote2 . In particular, MIREX allows for an objective assessment of the accuracy of different cover song identification algorithms. ..." in J. Serrà, X. Serra and R. G. Andrzejak. Cross recurrence quantification for cover song identification. New Journal of Physics, vol. 11, art. 093017. September 2009. http://mtg.upf.edu/node/1390

"MIREX is the widest participation contest in music information retrieval (MIR) field in the world. This contest holds annually and proposes new competitive tasks every year to keep up with the change in MIR field. The contest includes most research topics in MIR filed from traditional tasks such as Query by Singing/Humming task and Audio Genre Classification task to newest tasks such as Audio Music Mood Classification and Audio Tag Classification." Research News at Institute of Acoustics, Chinese Academy of Sciences, Nov. 2009, http://english.ioa.cas.cn/rh/as/200911/t20091106_46842.html

I have been working on music information retrieval for over 10 years and have proudly watched the field grow from a research curiosity to a vibrant research area with significant impact in both academia and industry. MIREX has been instrumental in this evolution. By providing a common evaluation framework and data it is possible to test new algorithms and get a much better understanding of the tradeoffs involved in their design. Every year after the ISMIR submissions are done I can't wait to get started working on MIREX submissions. Every year I try to participate in more tasks but due to time constraints only make it to a few. Running MIREX takes an incredible amount of effort and the organizers have done an outstanding job over the years. I sincerely hope that MIREX continuous to evolve and grow helping guide the growth of music information retrieval as a researcher area. George Tzanetakis, Associate Professor of Computer Science and Canada Research Chair in Computer Analysis of Music and Audio, University of Victoria, Canada.